2020-01-13 22:39:29,391 | root | WARNING | MainProcess | MainThread | [data_helper::31] Can not import concurrent module. Data for each server will be loaded/retrieved sequentially 2020-01-13 22:39:30,296 | test | INFO | MainProcess | MainThread | [basetestcase:_BaseTestCase__log_setup_status:385] ========= basetestcase setup started for test #1 test_add_back_failed_node ========= 2020-01-13 22:39:31,546 | test | INFO | MainProcess | MainThread | [basetestcase:_BaseTestCase__log_setup_status:385] ========= basetestcase setup started for test #1 test_add_back_failed_node ========= 2020-01-13 22:39:32,207 | test | ERROR | MainProcess | MainThread | [rest_client:_http_request:737] GET http://172.23.120.162:8091/pools/default/ body: headers: {'Accept': '*/*', 'Connection': 'close', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n', 'Content-Type': 'application/x-www-form-urlencoded'} error: 404 reason: unknown "unknown pool" auth: Administrator:password 2020-01-13 22:40:05,328 | test | INFO | MainProcess | MainThread | [basetestcase:setUp:251] Initializing cluster 2020-01-13 22:40:05,684 | test | WARNING | MainProcess | MainThread | [basetestcase:_initialize_nodes:416] RAM quota was defined less than 100 MB: 2020-01-13 22:40:09,005 | test | INFO | MainProcess | MainThread | [basetestcase:setUp:273] Couchbase Cluster: default, Nodes: 172.23.120.162, 172.23.120.177, 172.23.120.19, 172.23.120.192, 172.23.121.17, 172.23.121.19 initialized 2020-01-13 22:40:09,025 | test | INFO | MainProcess | MainThread | [basetestcase:_BaseTestCase__log_setup_status:385] ========= basetestcase setup finished for test #1 test_add_back_failed_node ========= 2020-01-13 22:40:09,375 | test | INFO | MainProcess | MainThread | [basetestcase:sleep:486] Reason: None. Sleep for 5 secs ... 2020-01-13 22:40:22,703 | test | INFO | MainProcess | pool-2-thread-7 | [table_view:display:72] Rebalance Overview +----------------+-----------------+--------------+ | Nodes | Services | Status | +----------------+-----------------+--------------+ | 172.23.120.162 | index, kv, n1ql | Cluster node | | 172.23.120.177 | None | <--- IN --- | | 172.23.120.19 | None | <--- IN --- | | 172.23.120.192 | None | <--- IN --- | +----------------+-----------------+--------------+ 2020-01-13 22:40:22,717 | test | INFO | MainProcess | pool-2-thread-7 | [task:check:317] Rebalance - status: running, progress: 0 2020-01-13 22:40:37,750 | test | INFO | MainProcess | pool-2-thread-7 | [task:check:317] Rebalance - status: none, progress: 100 2020-01-13 22:40:37,766 | test | INFO | MainProcess | pool-2-thread-7 | [task:check:376] Rebalance completed with progress: 100% in 15.0639998913 sec 2020-01-13 22:40:46,180 | test | INFO | MainProcess | MainThread | [rebalance_base:setUp:65] Verifying num_items counts after doc_ops 2020-01-13 22:40:48,818 | test | INFO | MainProcess | MainThread | [data_helper:direct_client:268] creating direct client 172.23.120.162:11210 default 2020-01-13 22:40:51,039 | test | INFO | MainProcess | MainThread | [data_helper:direct_client:268] creating direct client 172.23.120.177:11210 default 2020-01-13 22:40:53,125 | test | INFO | MainProcess | MainThread | [data_helper:direct_client:268] creating direct client 172.23.120.19:11210 default 2020-01-13 22:40:54,934 | test | INFO | MainProcess | MainThread | [data_helper:direct_client:268] creating direct client 172.23.120.192:11210 default 2020-01-13 22:41:07,441 | test | INFO | MainProcess | MainThread | [table_view:display:72] Cluster statistics +----------------+-----------------+-----------------+------------+------------+---------------------+-------------------+-----------------------+ | Node | Services | CPU_utilization | Mem_total | Mem_free | Swap_mem_used | Active / Replica | Version | +----------------+-----------------+-----------------+------------+------------+---------------------+-------------------+-----------------------+ | 172.23.120.192 | kv | 9.27835051546 | 3965001728 | 3231191040 | 0 / 3758092288 | 250 / 498 | 6.5.0-4960-enterprise | | 172.23.120.19 | kv | 3.28282828283 | 4220141568 | 3545063424 | 262144 / 3758092288 | 250 / 500 | 6.5.0-4960-enterprise | | 172.23.120.162 | index, kv, n1ql | 1.75438596491 | 3963588608 | 3175989248 | 0 / 2147479552 | 250 / 507 | 6.5.0-4960-enterprise | | 172.23.120.177 | kv | 1.75 | 3965001728 | 3205324800 | 0 / 3758092288 | 250 / 495 | 6.5.0-4960-enterprise | +----------------+-----------------+-----------------+------------+------------+---------------------+-------------------+-----------------------+ 2020-01-13 22:41:07,479 | test | INFO | MainProcess | MainThread | [table_view:display:72] Bucket statistics +---------+---------+----------+-----+-------+------------+----------+-----------+ | Bucket | Type | Replicas | TTL | Items | RAM Quota | RAM Used | Disk Used | +---------+---------+----------+-----+-------+------------+----------+-----------+ | default | membase | 2 | 0 | 1000 | 5632950272 | 90420000 | 29638097 | +---------+---------+----------+-----+-------+------------+----------+-----------+ 2020-01-13 22:41:07,480 | test | INFO | MainProcess | MainThread | [rebalance_base:setUp:85] ==========Finished rebalance base setup======== 2020-01-13 22:41:07,480 | test | INFO | MainProcess | MainThread | [swaprebalancetests:setUp:21] === SwapRebalanceBase setup started for test #1 test_add_back_failed_node === 2020-01-13 22:41:07,493 | test | INFO | MainProcess | MainThread | [swaprebalancetests:setUp:44] picking server : ip:172.23.120.162 port:8091 ssh_username:root as the master 2020-01-13 22:41:08,043 | test | INFO | MainProcess | MainThread | [swaprebalancetests:enable_diag_eval_on_non_local_hosts:90] Enabled diag/eval for non-local hosts from 172.23.120.162 2020-01-13 22:41:08,068 | test | INFO | MainProcess | MainThread | [swaprebalancetests:setUp:69] === SwapRebalanceBase setup done for test #1 test_add_back_failed_node === 2020-01-13 22:41:08,375 | test | INFO | MainProcess | MainThread | [swaprebalancetests:_add_back_failed_node:391] current nodes : [u'ns_1@172.23.120.192', u'ns_1@172.23.120.19', u'ns_1@172.23.120.162', u'ns_1@172.23.120.177'] 2020-01-13 22:41:08,427 | test | INFO | MainProcess | MainThread | [swaprebalancetests:_add_back_failed_node:396] To be removed nodes: [, ] 2020-01-13 22:41:08,428 | test | INFO | MainProcess | MainThread | [swaprebalancetests:_add_back_failed_node:409] Node 172.23.120.162:8091 not failed over 2020-01-13 22:41:08,428 | test | INFO | MainProcess | MainThread | [swaprebalancetests:_add_back_failed_node:409] Node 172.23.120.177:8091 not failed over 2020-01-13 22:41:08,430 | test | INFO | MainProcess | MainThread | [swaprebalancetests:_add_back_failed_node:423] To be removed nodes: [u'ns_1@172.23.120.192', u'ns_1@172.23.120.19'] 2020-01-13 22:41:08,430 | test | INFO | MainProcess | MainThread | [swaprebalancetests:_add_back_failed_node:426] Failover node ns_1@172.23.120.192 and rebalance afterwards 2020-01-13 22:41:09,177 | test | INFO | MainProcess | pool-2-thread-10 | [table_view:display:72] Failed items: +--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001068 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":10,"idempotent":false,"lastChannelId":"000000000000000A/000000005F4AFE10","lastDispatchedFrom":"172.23.97.100:59774","lastDispatchedTo":"172.23.120.192:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1631,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001068","opaque":"0x928","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":24,"totalMicros":382709}} | | test_docs-00001067 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":10,"idempotent":false,"lastChannelId":"000000000000000A/000000005F4AFE10","lastDispatchedFrom":"172.23.97.100:59774","lastDispatchedTo":"172.23.120.192:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1640,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001067","opaque":"0x931","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":30,"totalMicros":381910}} | +--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:41:09,177 | test | INFO | MainProcess | pool-2-thread-7 | [table_view:display:72] Failed items: +--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001325 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":12,"idempotent":false,"lastChannelId":"000000000000000C/0000000040B49B37","lastDispatchedFrom":"172.23.97.100:59804","lastDispatchedTo":"172.23.120.192:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1616,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001325","opaque":"0x919","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":22,"totalMicros":381121}} | +--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:41:09,177 | test | INFO | MainProcess | pool-2-thread-9 | [table_view:display:72] Failed items: +--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001212 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":11,"idempotent":false,"lastChannelId":"000000000000000B/00000000474D2BF1","lastDispatchedFrom":"172.23.97.100:59802","lastDispatchedTo":"172.23.120.192:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1706,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001212","opaque":"0x973","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":21,"totalMicros":351930}} | | test_docs-00001223 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":11,"idempotent":false,"lastChannelId":"000000000000000B/00000000474D2BF1","lastDispatchedFrom":"172.23.97.100:59802","lastDispatchedTo":"172.23.120.192:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1707,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001223","opaque":"0x974","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":24,"totalMicros":352067}} | | test_docs-00001217 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":11,"idempotent":false,"lastChannelId":"000000000000000B/00000000474D2BF1","lastDispatchedFrom":"172.23.97.100:59802","lastDispatchedTo":"172.23.120.192:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1721,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001217","opaque":"0x982","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":22,"totalMicros":348888}} | | test_docs-00001218 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":11,"idempotent":false,"lastChannelId":"000000000000000B/00000000474D2BF1","lastDispatchedFrom":"172.23.97.100:59802","lastDispatchedTo":"172.23.120.192:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1723,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001218","opaque":"0x984","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":23,"totalMicros":350567}} | +--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:41:09,209 | test | INFO | MainProcess | MainThread | [swaprebalancetests:_add_back_failed_node:426] Failover node ns_1@172.23.120.19 and rebalance afterwards 2020-01-13 22:41:09,431 | test | INFO | MainProcess | pool-2-thread-4 | [table_view:display:72] Failed items: +--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001576 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":14,"idempotent":false,"lastChannelId":"000000000000000E/00000000C0AA5DCD","lastDispatchedFrom":"172.23.97.100:59836","lastDispatchedTo":"172.23.120.192:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1612,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001576","opaque":"0x915","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":22,"totalMicros":381264}} | +--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:41:09,437 | test | INFO | MainProcess | pool-2-thread-5 | [table_view:display:72] Failed items: +--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001820 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":16,"idempotent":false,"lastChannelId":"0000000000000010/00000000DC2D4016","lastDispatchedFrom":"172.23.97.100:59862","lastDispatchedTo":"172.23.120.192:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1655,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001820","opaque":"0x940","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":20,"totalMicros":368760}} | | test_docs-00001814 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":16,"idempotent":false,"lastChannelId":"0000000000000010/00000000DC2D4016","lastDispatchedFrom":"172.23.97.100:59862","lastDispatchedTo":"172.23.120.192:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1656,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001814","opaque":"0x941","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":25,"totalMicros":371413}} | | test_docs-00001825 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":16,"idempotent":false,"lastChannelId":"0000000000000010/00000000DC2D4016","lastDispatchedFrom":"172.23.97.100:59862","lastDispatchedTo":"172.23.120.192:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1657,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001825","opaque":"0x942","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":25,"totalMicros":371551}} | | test_docs-00001811 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":16,"idempotent":false,"lastChannelId":"0000000000000010/00000000DC2D4016","lastDispatchedFrom":"172.23.97.100:59862","lastDispatchedTo":"172.23.120.192:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1662,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001811","opaque":"0x947","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":19,"totalMicros":371472}} | +--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:41:09,443 | test | INFO | MainProcess | pool-2-thread-3 | [table_view:display:72] Failed items: +--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001452 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":13,"idempotent":false,"lastChannelId":"000000000000000D/000000004AE6521C","lastDispatchedFrom":"172.23.97.100:59818","lastDispatchedTo":"172.23.120.192:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1687,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001452","opaque":"0x960","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":35,"totalMicros":366798}} | | test_docs-00001441 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":13,"idempotent":false,"lastChannelId":"000000000000000D/000000004AE6521C","lastDispatchedFrom":"172.23.97.100:59818","lastDispatchedTo":"172.23.120.192:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1685,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001441","opaque":"0x95e","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":21,"totalMicros":367711}} | | test_docs-00001435 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":13,"idempotent":false,"lastChannelId":"000000000000000D/000000004AE6521C","lastDispatchedFrom":"172.23.97.100:59818","lastDispatchedTo":"172.23.120.192:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1689,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001435","opaque":"0x962","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":28,"totalMicros":367059}} | | test_docs-00001444 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":13,"idempotent":false,"lastChannelId":"000000000000000D/000000004AE6521C","lastDispatchedFrom":"172.23.97.100:59818","lastDispatchedTo":"172.23.120.192:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1697,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001444","opaque":"0x96a","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":20,"totalMicros":366871}} | +--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:41:09,441 | test | INFO | MainProcess | pool-2-thread-8 | [table_view:display:72] Failed items: +--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001685 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":15,"idempotent":false,"lastChannelId":"000000000000000F/00000000F992EF23","lastDispatchedFrom":"172.23.97.100:59852","lastDispatchedTo":"172.23.120.192:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1667,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001685","opaque":"0x94c","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":25,"totalMicros":370756}} | | test_docs-00001696 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":15,"idempotent":false,"lastChannelId":"000000000000000F/00000000F992EF23","lastDispatchedFrom":"172.23.97.100:59852","lastDispatchedTo":"172.23.120.192:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1668,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001696","opaque":"0x94d","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":20,"totalMicros":370885}} | | test_docs-00001693 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":15,"idempotent":false,"lastChannelId":"000000000000000F/00000000F992EF23","lastDispatchedFrom":"172.23.97.100:59852","lastDispatchedTo":"172.23.120.192:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1673,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001693","opaque":"0x952","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":22,"totalMicros":367060}} | | test_docs-00001699 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":15,"idempotent":false,"lastChannelId":"000000000000000F/00000000F992EF23","lastDispatchedFrom":"172.23.97.100:59852","lastDispatchedTo":"172.23.120.192:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1680,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001699","opaque":"0x959","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":21,"totalMicros":369765}} | | test_docs-00001703 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":15,"idempotent":false,"lastChannelId":"000000000000000F/00000000F992EF23","lastDispatchedFrom":"172.23.97.100:59852","lastDispatchedTo":"172.23.120.192:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1691,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001703","opaque":"0x964","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":26,"totalMicros":368873}} | +--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:41:09,454 | test | INFO | MainProcess | pool-2-thread-2 | [table_view:display:72] Failed items: +--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001941 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":17,"idempotent":false,"lastChannelId":"0000000000000011/00000000A98C5D1A","lastDispatchedFrom":"172.23.97.100:59878","lastDispatchedTo":"172.23.120.192:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1733,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001941","opaque":"0x98e","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":18,"totalMicros":348158}} | | test_docs-00001952 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":17,"idempotent":false,"lastChannelId":"0000000000000011/00000000A98C5D1A","lastDispatchedFrom":"172.23.97.100:59878","lastDispatchedTo":"172.23.120.192:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1734,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001952","opaque":"0x98f","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":17,"totalMicros":348321}} | | test_docs-00001935 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":17,"idempotent":false,"lastChannelId":"0000000000000011/00000000A98C5D1A","lastDispatchedFrom":"172.23.97.100:59878","lastDispatchedTo":"172.23.120.192:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1738,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001935","opaque":"0x993","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":39,"totalMicros":347206}} | | test_docs-00001944 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":17,"idempotent":false,"lastChannelId":"0000000000000011/00000000A98C5D1A","lastDispatchedFrom":"172.23.97.100:59878","lastDispatchedTo":"172.23.120.192:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1741,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001944","opaque":"0x996","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":27,"totalMicros":347285}} | +--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:41:12,344 | test | INFO | MainProcess | pool-2-thread-10 | [table_view:display:72] Failed items: +--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001085 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":10,"idempotent":false,"lastChannelId":"000000000000000A/000000007A54F6D5","lastDispatchedFrom":"172.23.97.100:34694","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1828,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001085","opaque":"0xaad","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":11,"totalMicros":531883}} | | test_docs-00001096 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":10,"idempotent":false,"lastChannelId":"000000000000000A/000000007A54F6D5","lastDispatchedFrom":"172.23.97.100:34694","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1829,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001096","opaque":"0xaae","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":9,"totalMicros":532083}} | | test_docs-00001089 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":10,"idempotent":false,"lastChannelId":"000000000000000A/000000007A54F6D5","lastDispatchedFrom":"172.23.97.100:34694","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1832,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001089","opaque":"0xab1","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":10,"totalMicros":531456}} | +--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:41:12,345 | test | INFO | MainProcess | pool-2-thread-8 | [table_view:display:72] Failed items: +--------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001719 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":15,"idempotent":false,"lastChannelId":"000000000000000F/00000000E95D4D0A","lastDispatchedFrom":"172.23.97.100:34766","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1958,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001719","opaque":"0xb51","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":12,"totalMicros":271328}} | | test_docs-00001706 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":15,"idempotent":false,"lastChannelId":"000000000000000F/00000000EF0816A7","lastDispatchedFrom":"172.23.97.100:37774","lastDispatchedTo":"172.23.120.162:11210","requestId":1961,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001706","opaque":"0xb54","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":5829,"encodingMicros":10,"totalMicros":8008,"serverMicros":0}} | | test_docs-00001705 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":15,"idempotent":false,"lastChannelId":"000000000000000F/00000000E95D4D0A","lastDispatchedFrom":"172.23.97.100:34766","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1963,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001705","opaque":"0xb56","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":7,"totalMicros":271315}} | | test_docs-00001716 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":15,"idempotent":false,"lastChannelId":"000000000000000F/00000000E95D4D0A","lastDispatchedFrom":"172.23.97.100:34766","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1964,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001716","opaque":"0xb57","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":7,"totalMicros":271504}} | | test_docs-00001709 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":15,"idempotent":false,"lastChannelId":"000000000000000F/00000000E95D4D0A","lastDispatchedFrom":"172.23.97.100:34766","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1965,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001709","opaque":"0xb58","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":13,"totalMicros":271734}} | | test_docs-00001722 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":15,"idempotent":false,"lastChannelId":"000000000000000F/00000000E95D4D0A","lastDispatchedFrom":"172.23.97.100:34766","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1967,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001722","opaque":"0xb5a","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":16,"totalMicros":271881}} | | test_docs-00001710 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":15,"idempotent":false,"lastChannelId":"000000000000000F/00000000EF0816A7","lastDispatchedFrom":"172.23.97.100:37774","lastDispatchedTo":"172.23.120.162:11210","requestId":1968,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001710","opaque":"0xb5b","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":6565,"encodingMicros":72,"totalMicros":8496,"serverMicros":0}} | | test_docs-00001713 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":15,"idempotent":false,"lastChannelId":"000000000000000F/00000000E95D4D0A","lastDispatchedFrom":"172.23.97.100:34766","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1973,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001713","opaque":"0xb60","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":7,"totalMicros":268559}} | +--------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:41:12,348 | test | INFO | MainProcess | pool-2-thread-7 | [table_view:display:72] Failed items: +--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001366 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":12,"idempotent":false,"lastChannelId":"000000000000000C/000000000E9CDBE3","lastDispatchedFrom":"172.23.97.100:34716","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1873,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001366","opaque":"0xada","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":17,"totalMicros":449753}} | | test_docs-00001352 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":12,"idempotent":false,"lastChannelId":"000000000000000C/000000000E9CDBE3","lastDispatchedFrom":"172.23.97.100:34716","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1874,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001352","opaque":"0xadb","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":14,"totalMicros":449854}} | | test_docs-00001363 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":12,"idempotent":false,"lastChannelId":"000000000000000C/000000000E9CDBE3","lastDispatchedFrom":"172.23.97.100:34716","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1875,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001363","opaque":"0xadc","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":12,"totalMicros":449941}} | | test_docs-00001369 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":12,"idempotent":false,"lastChannelId":"000000000000000C/000000000E9CDBE3","lastDispatchedFrom":"172.23.97.100:34716","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1879,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001369","opaque":"0xae0","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":17,"totalMicros":448970}} | | test_docs-00001358 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":12,"idempotent":false,"lastChannelId":"000000000000000C/000000000E9CDBE3","lastDispatchedFrom":"172.23.97.100:34716","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1878,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001358","opaque":"0xadf","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":17,"totalMicros":449931}} | | test_docs-00001357 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":12,"idempotent":false,"lastChannelId":"000000000000000C/000000000E9CDBE3","lastDispatchedFrom":"172.23.97.100:34716","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1883,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001357","opaque":"0xae4","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":13,"totalMicros":449010}} | +--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:41:12,348 | test | INFO | MainProcess | pool-2-thread-2 | [table_view:display:72] Failed items: +--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001969 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":17,"idempotent":false,"lastChannelId":"0000000000000011/000000005F5B018E","lastDispatchedFrom":"172.23.97.100:37806","lastDispatchedTo":"172.23.120.162:11210","requestId":1979,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001969","opaque":"0xb68","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":4769,"encodingMicros":12,"totalMicros":4960,"serverMicros":0}} | | test_docs-00001960 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":17,"idempotent":false,"lastChannelId":"0000000000000011/0000000065AC2F35","lastDispatchedFrom":"172.23.97.100:34794","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1980,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001960","opaque":"0xb69","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":8,"totalMicros":260460}} | | test_docs-00001970 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":17,"idempotent":false,"lastChannelId":"0000000000000011/0000000065AC2F35","lastDispatchedFrom":"172.23.97.100:34794","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1982,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001970","opaque":"0xb6b","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":5,"totalMicros":260963}} | | test_docs-00001964 | com.couchbase.client.core.error.DurabilityAmbiguousException: The server returned with a durability ambiguous response on this request {"completed":true,"coreId":17,"idempotent":false,"lastChannelId":"0000000000000011/00000000B749AAAB","lastDispatchedFrom":"172.23.97.100:42262","lastDispatchedTo":"172.23.120.177:11210","requestId":1983,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001964","opaque":"0xb6c","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"SYNC_WRITE_AMBIGUOUS","timeoutMs":5000,"timings":{"dispatchMicros":45031,"encodingMicros":5,"totalMicros":45327,"serverMicros":44}} | | test_docs-00001963 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":17,"idempotent":false,"lastChannelId":"0000000000000011/0000000065AC2F35","lastDispatchedFrom":"172.23.97.100:34794","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1984,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001963","opaque":"0xb6d","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":6,"totalMicros":261203}} | | test_docs-00001973 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":17,"idempotent":false,"lastChannelId":"0000000000000011/0000000065AC2F35","lastDispatchedFrom":"172.23.97.100:34794","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1987,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001973","opaque":"0xb70","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":8,"totalMicros":259762}} | | test_docs-00001961 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":17,"idempotent":false,"lastChannelId":"0000000000000011/00000000B749AAAB","lastDispatchedFrom":"172.23.97.100:42262","lastDispatchedTo":"172.23.120.177:11210","requestId":1988,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001961","opaque":"0xb71","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":45178,"encodingMicros":6,"totalMicros":45313,"serverMicros":0}} | | test_docs-00001972 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":17,"idempotent":false,"lastChannelId":"0000000000000011/00000000B749AAAB","lastDispatchedFrom":"172.23.97.100:42262","lastDispatchedTo":"172.23.120.177:11210","requestId":1989,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001972","opaque":"0xb72","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":45280,"encodingMicros":6,"totalMicros":45412,"serverMicros":0}} | | test_docs-00001957 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":17,"idempotent":false,"lastChannelId":"0000000000000011/0000000065AC2F35","lastDispatchedFrom":"172.23.97.100:34794","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1990,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001957","opaque":"0xb73","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":5,"totalMicros":259964}} | | test_docs-00001955 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":17,"idempotent":false,"lastChannelId":"0000000000000011/00000000B749AAAB","lastDispatchedFrom":"172.23.97.100:42262","lastDispatchedTo":"172.23.120.177:11210","requestId":1994,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001955","opaque":"0xb77","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":45240,"encodingMicros":10,"totalMicros":45375,"serverMicros":0}} | | test_docs-00001966 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":17,"idempotent":false,"lastChannelId":"0000000000000011/00000000B749AAAB","lastDispatchedFrom":"172.23.97.100:42262","lastDispatchedTo":"172.23.120.177:11210","requestId":1995,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001966","opaque":"0xb78","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":45313,"encodingMicros":8,"totalMicros":45455,"serverMicros":0}} | | test_docs-00001965 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":17,"idempotent":false,"lastChannelId":"0000000000000011/0000000065AC2F35","lastDispatchedFrom":"172.23.97.100:34794","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1996,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001965","opaque":"0xb79","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":5,"totalMicros":260282}} | +--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:41:12,352 | test | INFO | MainProcess | pool-2-thread-9 | [table_view:display:72] Failed items: +--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001225 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":11,"idempotent":false,"lastChannelId":"000000000000000B/00000000325B69B2","lastDispatchedFrom":"172.23.97.100:34712","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1848,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001225","opaque":"0xac1","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":17,"totalMicros":531077}} | | test_docs-00001236 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":11,"idempotent":false,"lastChannelId":"000000000000000B/00000000325B69B2","lastDispatchedFrom":"172.23.97.100:34712","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1849,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001236","opaque":"0xac2","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":16,"totalMicros":531246}} | | test_docs-00001239 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":11,"idempotent":false,"lastChannelId":"000000000000000B/00000000325B69B2","lastDispatchedFrom":"172.23.97.100:34712","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1851,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001239","opaque":"0xac4","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":17,"totalMicros":531345}} | +--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:41:12,357 | test | INFO | MainProcess | pool-2-thread-3 | [table_view:display:72] Failed items: +--------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001460 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":13,"idempotent":false,"lastChannelId":"000000000000000D/00000000837FE64C","lastDispatchedFrom":"172.23.97.100:34738","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1919,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001460","opaque":"0xb28","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":25,"totalMicros":275806}} | | test_docs-00001465 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":13,"idempotent":false,"lastChannelId":"000000000000000D/00000000837FE64C","lastDispatchedFrom":"172.23.97.100:34738","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1932,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001465","opaque":"0xb35","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":12,"totalMicros":275117}} | | test_docs-00001473 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":13,"idempotent":false,"lastChannelId":"000000000000000D/00000000837FE64C","lastDispatchedFrom":"172.23.97.100:34738","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1936,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001473","opaque":"0xb39","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":9,"totalMicros":275668}} | | test_docs-00001463 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":13,"idempotent":false,"lastChannelId":"000000000000000D/00000000F2B69A35","lastDispatchedFrom":"172.23.97.100:37736","lastDispatchedTo":"172.23.120.162:11210","requestId":1938,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001463","opaque":"0xb3b","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":8085,"encodingMicros":12,"totalMicros":9026,"serverMicros":0}} | | test_docs-00001458 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":13,"idempotent":false,"lastChannelId":"000000000000000D/00000000837FE64C","lastDispatchedFrom":"172.23.97.100:34738","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1946,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001458","opaque":"0xb43","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":15,"totalMicros":274693}} | | test_docs-00001469 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":13,"idempotent":false,"lastChannelId":"000000000000000D/00000000837FE64C","lastDispatchedFrom":"172.23.97.100:34738","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1948,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001469","opaque":"0xb45","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":61,"totalMicros":274898}} | +--------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:41:12,358 | test | INFO | MainProcess | pool-2-thread-4 | [table_view:display:72] Failed items: +--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001581 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":14,"idempotent":false,"lastChannelId":"000000000000000E/0000000032B46F69","lastDispatchedFrom":"172.23.97.100:34756","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1899,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001581","opaque":"0xb14","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":9,"totalMicros":277001}} | | test_docs-00001592 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":14,"idempotent":false,"lastChannelId":"000000000000000E/0000000032B46F69","lastDispatchedFrom":"172.23.97.100:34756","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1900,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001592","opaque":"0xb15","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":11,"totalMicros":277279}} | | test_docs-00001597 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":14,"idempotent":false,"lastChannelId":"000000000000000E/0000000032B46F69","lastDispatchedFrom":"172.23.97.100:34756","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1905,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001597","opaque":"0xb1a","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":36,"totalMicros":277420}} | | test_docs-00001584 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":14,"idempotent":false,"lastChannelId":"000000000000000E/0000000032B46F69","lastDispatchedFrom":"172.23.97.100:34756","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1908,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001584","opaque":"0xb1d","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":17,"totalMicros":277431}} | | test_docs-00001588 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":14,"idempotent":false,"lastChannelId":"000000000000000E/00000000C4B583FE","lastDispatchedFrom":"172.23.97.100:42214","lastDispatchedTo":"172.23.120.177:11210","requestId":1913,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001588","opaque":"0xb22","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":15199,"encodingMicros":13,"totalMicros":15580,"serverMicros":0}} | | test_docs-00001598 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":14,"idempotent":false,"lastChannelId":"000000000000000E/0000000032B46F69","lastDispatchedFrom":"172.23.97.100:34756","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1916,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001598","opaque":"0xb25","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":6,"totalMicros":276826}} | +--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:41:12,358 | test | INFO | MainProcess | pool-2-thread-5 | [table_view:display:72] Failed items: +--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001839 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":16,"idempotent":false,"lastChannelId":"0000000000000010/00000000A0717028","lastDispatchedFrom":"172.23.97.100:42240","lastDispatchedTo":"172.23.120.177:11210","requestId":1917,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001839","opaque":"0xb26","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":472,"encodingMicros":39,"totalMicros":1799,"serverMicros":0}} | | test_docs-00001848 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":16,"idempotent":false,"lastChannelId":"0000000000000010/00000000BABFD5E4","lastDispatchedFrom":"172.23.97.100:34778","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1923,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001848","opaque":"0xb2c","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":10,"totalMicros":275022}} | | test_docs-00001830 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":16,"idempotent":false,"lastChannelId":"0000000000000010/00000000BABFD5E4","lastDispatchedFrom":"172.23.97.100:34778","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1933,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001830","opaque":"0xb36","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":13,"totalMicros":274009}} | | test_docs-00001841 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":16,"idempotent":false,"lastChannelId":"0000000000000010/00000000BABFD5E4","lastDispatchedFrom":"172.23.97.100:34778","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1935,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001841","opaque":"0xb38","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":11,"totalMicros":274239}} | | test_docs-00001836 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":16,"idempotent":false,"lastChannelId":"0000000000000010/00000000C5A7DA7C","lastDispatchedFrom":"172.23.97.100:37800","lastDispatchedTo":"172.23.120.162:11210","requestId":1939,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001836","opaque":"0xb3c","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":13920,"encodingMicros":10,"totalMicros":14341,"serverMicros":0}} | | test_docs-00001835 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":16,"idempotent":false,"lastChannelId":"0000000000000010/00000000BABFD5E4","lastDispatchedFrom":"172.23.97.100:34778","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1943,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001835","opaque":"0xb40","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":11,"totalMicros":273533}} | | test_docs-00001844 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":16,"idempotent":false,"lastChannelId":"0000000000000010/00000000BABFD5E4","lastDispatchedFrom":"172.23.97.100:34778","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":1953,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001844","opaque":"0xb4a","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":13,"totalMicros":274103}} | +--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:41:12,391 | test | INFO | MainProcess | pool-2-thread-9 | [table_view:display:72] Failed items: +--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001245 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":11,"idempotent":false,"lastChannelId":"000000000000000B/0000000065519A5B","lastDispatchedFrom":"172.23.97.100:37710","lastDispatchedTo":"172.23.120.162:11210","requestId":2173,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001245","opaque":"0xda2","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":3423,"encodingMicros":15,"totalMicros":3738,"serverMicros":0}} | | test_docs-00001246 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":11,"idempotent":false,"lastChannelId":"000000000000000B/00000000C361FFC0","lastDispatchedFrom":"172.23.97.100:42178","lastDispatchedTo":"172.23.120.177:11210","requestId":2176,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001246","opaque":"0xda5","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":15342,"encodingMicros":5,"totalMicros":15497,"serverMicros":0}} | | test_docs-00001247 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":11,"idempotent":false,"lastChannelId":"000000000000000B/00000000C361FFC0","lastDispatchedFrom":"172.23.97.100:42178","lastDispatchedTo":"172.23.120.177:11210","requestId":2177,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001247","opaque":"0xda6","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":15425,"encodingMicros":5,"totalMicros":15551,"serverMicros":0}} | +--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:41:12,404 | test | INFO | MainProcess | pool-2-thread-10 | [table_view:display:72] Failed items: +--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001101 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":10,"idempotent":false,"lastChannelId":"000000000000000A/0000000017CACF8F","lastDispatchedFrom":"172.23.97.100:42154","lastDispatchedTo":"172.23.120.177:11210","requestId":2128,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001101","opaque":"0xd75","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":1732,"encodingMicros":37,"totalMicros":2472,"serverMicros":0}} | | test_docs-00001102 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":10,"idempotent":false,"lastChannelId":"000000000000000A/00000000F31F61A3","lastDispatchedFrom":"172.23.97.100:37706","lastDispatchedTo":"172.23.120.162:11210","requestId":2130,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001102","opaque":"0xd77","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":3831,"encodingMicros":6,"totalMicros":4736,"serverMicros":0}} | | test_docs-00001113 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":10,"idempotent":false,"lastChannelId":"000000000000000A/00000000F31F61A3","lastDispatchedFrom":"172.23.97.100:37706","lastDispatchedTo":"172.23.120.162:11210","requestId":2131,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001113","opaque":"0xd78","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":3966,"encodingMicros":6,"totalMicros":4844,"serverMicros":0}} | | test_docs-00001100 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":10,"idempotent":false,"lastChannelId":"000000000000000A/00000000F31F61A3","lastDispatchedFrom":"172.23.97.100:37706","lastDispatchedTo":"172.23.120.162:11210","requestId":2133,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001100","opaque":"0xd7a","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":7919,"encodingMicros":5,"totalMicros":8801,"serverMicros":0}} | | test_docs-00001116 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":10,"idempotent":false,"lastChannelId":"000000000000000A/0000000017CACF8F","lastDispatchedFrom":"172.23.97.100:42154","lastDispatchedTo":"172.23.120.177:11210","requestId":2136,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001116","opaque":"0xd7d","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":11083,"encodingMicros":4,"totalMicros":11609,"serverMicros":0}} | | test_docs-00001105 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":10,"idempotent":false,"lastChannelId":"000000000000000A/00000000F31F61A3","lastDispatchedFrom":"172.23.97.100:37706","lastDispatchedTo":"172.23.120.162:11210","requestId":2135,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001105","opaque":"0xd7c","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":17105,"encodingMicros":4,"totalMicros":17936,"serverMicros":0}} | | test_docs-00001114 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":10,"idempotent":false,"lastChannelId":"000000000000000A/00000000F31F61A3","lastDispatchedFrom":"172.23.97.100:37706","lastDispatchedTo":"172.23.120.162:11210","requestId":2140,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001114","opaque":"0xd81","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":20936,"encodingMicros":4,"totalMicros":21707,"serverMicros":0}} | +--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:41:12,404 | test | INFO | MainProcess | pool-2-thread-4 | [table_view:display:72] Failed items: +--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001609 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":14,"idempotent":false,"lastChannelId":"000000000000000E/000000003A488558","lastDispatchedFrom":"172.23.97.100:37764","lastDispatchedTo":"172.23.120.162:11210","requestId":2218,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001609","opaque":"0xdcf","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":494,"encodingMicros":13,"totalMicros":646,"serverMicros":0}} | | test_docs-00001606 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":14,"idempotent":false,"lastChannelId":"000000000000000E/00000000C4B583FE","lastDispatchedFrom":"172.23.97.100:42214","lastDispatchedTo":"172.23.120.177:11210","requestId":2223,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001606","opaque":"0xdd4","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":8406,"encodingMicros":5,"totalMicros":8526,"serverMicros":0}} | | test_docs-00001617 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":14,"idempotent":false,"lastChannelId":"000000000000000E/000000003A488558","lastDispatchedFrom":"172.23.97.100:37764","lastDispatchedTo":"172.23.120.162:11210","requestId":2224,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001617","opaque":"0xdd5","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":11009,"encodingMicros":5,"totalMicros":11140,"serverMicros":0}} | | test_docs-00001601 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":14,"idempotent":false,"lastChannelId":"000000000000000E/000000003A488558","lastDispatchedFrom":"172.23.97.100:37764","lastDispatchedTo":"172.23.120.162:11210","requestId":2225,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001601","opaque":"0xdd6","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":11140,"encodingMicros":5,"totalMicros":11234,"serverMicros":0}} | | test_docs-00001612 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":14,"idempotent":false,"lastChannelId":"000000000000000E/000000003A488558","lastDispatchedFrom":"172.23.97.100:37764","lastDispatchedTo":"172.23.120.162:11210","requestId":2226,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001612","opaque":"0xdd7","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":11194,"encodingMicros":7,"totalMicros":11282,"serverMicros":0}} | | test_docs-00001611 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":14,"idempotent":false,"lastChannelId":"000000000000000E/00000000C4B583FE","lastDispatchedFrom":"172.23.97.100:42214","lastDispatchedTo":"172.23.120.177:11210","requestId":2228,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001611","opaque":"0xdd9","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":12720,"encodingMicros":5,"totalMicros":12804,"serverMicros":0}} | | test_docs-00001604 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":14,"idempotent":false,"lastChannelId":"000000000000000E/000000003A488558","lastDispatchedFrom":"172.23.97.100:37764","lastDispatchedTo":"172.23.120.162:11210","requestId":2232,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001604","opaque":"0xddd","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":11264,"encodingMicros":5,"totalMicros":11375,"serverMicros":0}} | | test_docs-00001615 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":14,"idempotent":false,"lastChannelId":"000000000000000E/00000000C4B583FE","lastDispatchedFrom":"172.23.97.100:42214","lastDispatchedTo":"172.23.120.177:11210","requestId":2233,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001615","opaque":"0xdde","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":28871,"encodingMicros":5,"totalMicros":28996,"serverMicros":0}} | | test_docs-00001603 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":14,"idempotent":false,"lastChannelId":"000000000000000E/00000000C4B583FE","lastDispatchedFrom":"172.23.97.100:42214","lastDispatchedTo":"172.23.120.177:11210","requestId":2234,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001603","opaque":"0xddf","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":29021,"encodingMicros":5,"totalMicros":29119,"serverMicros":0}} | | test_docs-00001614 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":14,"idempotent":false,"lastChannelId":"000000000000000E/00000000C4B583FE","lastDispatchedFrom":"172.23.97.100:42214","lastDispatchedTo":"172.23.120.177:11210","requestId":2235,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001614","opaque":"0xde0","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":29095,"encodingMicros":4,"totalMicros":29179,"serverMicros":0}} | | test_docs-00001602 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":14,"idempotent":false,"lastChannelId":"000000000000000E/00000000C4B583FE","lastDispatchedFrom":"172.23.97.100:42214","lastDispatchedTo":"172.23.120.177:11210","requestId":2236,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001602","opaque":"0xde1","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":29168,"encodingMicros":5,"totalMicros":29254,"serverMicros":0}} | +--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:41:12,417 | test | INFO | MainProcess | pool-2-thread-4 | [table_view:display:72] Failed items: +--------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001623 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":14,"idempotent":false,"lastChannelId":"000000000000000E/000000003A488558","lastDispatchedFrom":"172.23.97.100:37764","lastDispatchedTo":"172.23.120.162:11210","requestId":2284,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001623","opaque":"0xe25","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":1253,"encodingMicros":14,"totalMicros":1507,"serverMicros":0}} | | test_docs-00001621 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":14,"idempotent":false,"lastChannelId":"000000000000000E/000000003A488558","lastDispatchedFrom":"172.23.97.100:37764","lastDispatchedTo":"172.23.120.162:11210","requestId":2286,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001621","opaque":"0xe27","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":3214,"encodingMicros":6,"totalMicros":3411,"serverMicros":0}} | | test_docs-00001620 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":14,"idempotent":false,"lastChannelId":"000000000000000E/00000000C4B583FE","lastDispatchedFrom":"172.23.97.100:42214","lastDispatchedTo":"172.23.120.177:11210","requestId":2287,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001620","opaque":"0xe28","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":521,"encodingMicros":5,"totalMicros":696,"serverMicros":0}} | | test_docs-00001624 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":14,"idempotent":false,"lastChannelId":"000000000000000E/000000003A488558","lastDispatchedFrom":"172.23.97.100:37764","lastDispatchedTo":"172.23.120.162:11210","requestId":2288,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001624","opaque":"0xe29","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":3285,"encodingMicros":6,"totalMicros":3422,"serverMicros":0}} | +--------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:41:12,417 | test | INFO | MainProcess | pool-2-thread-3 | [table_view:display:72] Failed items: +--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001490 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":13,"idempotent":false,"lastChannelId":"000000000000000D/0000000042A49A26","lastDispatchedFrom":"172.23.97.100:42198","lastDispatchedTo":"172.23.120.177:11210","requestId":2198,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001490","opaque":"0xdbb","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":595,"encodingMicros":14,"totalMicros":827,"serverMicros":0}} | | test_docs-00001482 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":13,"idempotent":false,"lastChannelId":"000000000000000D/00000000F2B69A35","lastDispatchedFrom":"172.23.97.100:37736","lastDispatchedTo":"172.23.120.162:11210","requestId":2199,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001482","opaque":"0xdbc","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":1142,"encodingMicros":10,"totalMicros":1365,"serverMicros":0}} | | test_docs-00001494 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":13,"idempotent":false,"lastChannelId":"000000000000000D/00000000F2B69A35","lastDispatchedFrom":"172.23.97.100:37736","lastDispatchedTo":"172.23.120.162:11210","requestId":2202,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001494","opaque":"0xdbf","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":6764,"encodingMicros":6,"totalMicros":6943,"serverMicros":0}} | | test_docs-00001491 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":13,"idempotent":false,"lastChannelId":"000000000000000D/0000000042A49A26","lastDispatchedFrom":"172.23.97.100:42198","lastDispatchedTo":"172.23.120.177:11210","requestId":2204,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001491","opaque":"0xdc1","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":8180,"encodingMicros":5,"totalMicros":9009,"serverMicros":0}} | | test_docs-00001486 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":13,"idempotent":false,"lastChannelId":"000000000000000D/0000000042A49A26","lastDispatchedFrom":"172.23.97.100:42198","lastDispatchedTo":"172.23.120.177:11210","requestId":2208,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001486","opaque":"0xdc5","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":19913,"encodingMicros":4,"totalMicros":20708,"serverMicros":0}} | | test_docs-00001476 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":13,"idempotent":false,"lastChannelId":"000000000000000D/00000000F2B69A35","lastDispatchedFrom":"172.23.97.100:37736","lastDispatchedTo":"172.23.120.162:11210","requestId":2209,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001476","opaque":"0xdc6","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":11910,"encodingMicros":5,"totalMicros":12000,"serverMicros":0}} | | test_docs-00001487 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":13,"idempotent":false,"lastChannelId":"000000000000000D/0000000042A49A26","lastDispatchedFrom":"172.23.97.100:42198","lastDispatchedTo":"172.23.120.177:11210","requestId":2210,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001487","opaque":"0xdc7","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":20012,"encodingMicros":5,"totalMicros":20773,"serverMicros":0}} | | test_docs-00001485 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":13,"idempotent":false,"lastChannelId":"000000000000000D/00000000F2B69A35","lastDispatchedFrom":"172.23.97.100:37736","lastDispatchedTo":"172.23.120.162:11210","requestId":2212,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001485","opaque":"0xdc9","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":14251,"encodingMicros":5,"totalMicros":14359,"serverMicros":0}} | | test_docs-00001478 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":13,"idempotent":false,"lastChannelId":"000000000000000D/0000000042A49A26","lastDispatchedFrom":"172.23.97.100:42198","lastDispatchedTo":"172.23.120.177:11210","requestId":2216,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001478","opaque":"0xdcd","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":42483,"encodingMicros":7,"totalMicros":43230,"serverMicros":0}} | +--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:41:12,427 | test | INFO | MainProcess | pool-2-thread-10 | [table_view:display:72] Failed items: +--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001120 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":10,"idempotent":false,"lastChannelId":"000000000000000A/00000000F31F61A3","lastDispatchedFrom":"172.23.97.100:37706","lastDispatchedTo":"172.23.120.162:11210","requestId":2279,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001120","opaque":"0xe20","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":406,"encodingMicros":20,"totalMicros":650,"serverMicros":0}} | | test_docs-00001123 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":10,"idempotent":false,"lastChannelId":"000000000000000A/0000000017CACF8F","lastDispatchedFrom":"172.23.97.100:42154","lastDispatchedTo":"172.23.120.177:11210","requestId":2280,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001123","opaque":"0xe21","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":512,"encodingMicros":7,"totalMicros":723,"serverMicros":0}} | | test_docs-00001122 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":10,"idempotent":false,"lastChannelId":"000000000000000A/0000000017CACF8F","lastDispatchedFrom":"172.23.97.100:42154","lastDispatchedTo":"172.23.120.177:11210","requestId":2283,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001122","opaque":"0xe24","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":13740,"encodingMicros":4,"totalMicros":13937,"serverMicros":0}} | +--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:41:12,427 | test | INFO | MainProcess | pool-2-thread-5 | [table_view:display:72] Failed items: +--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001859 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":16,"idempotent":false,"lastChannelId":"0000000000000010/00000000A0717028","lastDispatchedFrom":"172.23.97.100:42240","lastDispatchedTo":"172.23.120.177:11210","requestId":2238,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001859","opaque":"0xde4","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":1392,"encodingMicros":20,"totalMicros":1828,"serverMicros":0}} | | test_docs-00001863 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":16,"idempotent":false,"lastChannelId":"0000000000000010/00000000C5A7DA7C","lastDispatchedFrom":"172.23.97.100:37800","lastDispatchedTo":"172.23.120.162:11210","requestId":2247,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001863","opaque":"0xded","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":18950,"encodingMicros":5,"totalMicros":19163,"serverMicros":0}} | | test_docs-00001858 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":16,"idempotent":false,"lastChannelId":"0000000000000010/00000000C5A7DA7C","lastDispatchedFrom":"172.23.97.100:37800","lastDispatchedTo":"172.23.120.162:11210","requestId":2250,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001858","opaque":"0xdf0","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":22723,"encodingMicros":5,"totalMicros":22930,"serverMicros":0}} | | test_docs-00001868 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":16,"idempotent":false,"lastChannelId":"0000000000000010/00000000A0717028","lastDispatchedFrom":"172.23.97.100:42240","lastDispatchedTo":"172.23.120.177:11210","requestId":2253,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001868","opaque":"0xdf3","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":41083,"encodingMicros":5,"totalMicros":41321,"serverMicros":0}} | | test_docs-00001866 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":16,"idempotent":false,"lastChannelId":"0000000000000010/00000000C5A7DA7C","lastDispatchedFrom":"172.23.97.100:37800","lastDispatchedTo":"172.23.120.162:11210","requestId":2257,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001866","opaque":"0xdf7","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":25543,"encodingMicros":19,"totalMicros":25698,"serverMicros":0}} | +--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:41:12,427 | test | ERROR | MainProcess | pool-2-thread-1 | [task:call:1483] Failed to load 15 docs from 1000 to 1125 2020-01-13 22:41:12,428 | test | ERROR | MainProcess | pool-2-thread-1 | [task:call:1483] Failed to load 10 docs from 1125 to 1250 2020-01-13 22:41:12,428 | test | ERROR | MainProcess | pool-2-thread-1 | [task:call:1483] Failed to load 7 docs from 1250 to 1375 2020-01-13 22:41:12,431 | test | INFO | MainProcess | pool-2-thread-8 | [table_view:display:72] Failed items: +--------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001728 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":15,"idempotent":false,"lastChannelId":"000000000000000F/00000000EF0816A7","lastDispatchedFrom":"172.23.97.100:37774","lastDispatchedTo":"172.23.120.162:11210","requestId":2154,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001728","opaque":"0xd8f","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":2181,"encodingMicros":7,"totalMicros":2500,"serverMicros":0}} | +--------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:41:12,437 | test | ERROR | MainProcess | pool-2-thread-1 | [task:call:1483] Failed to load 19 docs from 1375 to 1500 2020-01-13 22:41:12,437 | test | ERROR | MainProcess | pool-2-thread-1 | [task:call:1483] Failed to load 22 docs from 1500 to 1625 2020-01-13 22:41:12,438 | test | INFO | MainProcess | pool-2-thread-2 | [table_view:display:72] Failed items: +--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001988 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":17,"idempotent":false,"lastChannelId":"0000000000000011/000000005F5B018E","lastDispatchedFrom":"172.23.97.100:37806","lastDispatchedTo":"172.23.120.162:11210","requestId":2195,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001988","opaque":"0xdb8","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":24882,"encodingMicros":5,"totalMicros":25008,"serverMicros":0}} | | test_docs-00001977 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":17,"idempotent":false,"lastChannelId":"0000000000000011/00000000B749AAAB","lastDispatchedFrom":"172.23.97.100:42262","lastDispatchedTo":"172.23.120.177:11210","requestId":2194,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001977","opaque":"0xdb7","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":68274,"encodingMicros":4,"totalMicros":68409,"serverMicros":0}} | | test_docs-00001976 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":17,"idempotent":false,"lastChannelId":"0000000000000011/00000000B749AAAB","lastDispatchedFrom":"172.23.97.100:42262","lastDispatchedTo":"172.23.120.177:11210","requestId":2196,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001976","opaque":"0xdb9","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":68375,"encodingMicros":5,"totalMicros":68465,"serverMicros":0}} | +--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:41:12,447 | test | INFO | MainProcess | pool-2-thread-5 | [table_view:display:72] Failed items: +--------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001870 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":16,"idempotent":false,"lastChannelId":"0000000000000010/00000000C5A7DA7C","lastDispatchedFrom":"172.23.97.100:37800","lastDispatchedTo":"172.23.120.162:11210","requestId":2317,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001870","opaque":"0xe46","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":9024,"encodingMicros":5,"totalMicros":9170,"serverMicros":0}} | +--------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:41:12,450 | test | INFO | MainProcess | pool-2-thread-8 | [table_view:display:72] Failed items: +--------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001748 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":15,"idempotent":false,"lastChannelId":"000000000000000F/00000000EF0816A7","lastDispatchedFrom":"172.23.97.100:37774","lastDispatchedTo":"172.23.120.162:11210","requestId":2325,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001748","opaque":"0xe4e","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":404,"encodingMicros":11,"totalMicros":588,"serverMicros":0}} | +--------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:41:12,450 | test | ERROR | MainProcess | pool-2-thread-1 | [task:call:1483] Failed to load 15 docs from 1625 to 1750 2020-01-13 22:41:12,450 | test | ERROR | MainProcess | pool-2-thread-1 | [task:call:1483] Failed to load 17 docs from 1750 to 1875 2020-01-13 22:41:12,463 | test | INFO | MainProcess | pool-2-thread-2 | [table_view:display:72] Failed items: +--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001999 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":17,"idempotent":false,"lastChannelId":"0000000000000011/000000005F5B018E","lastDispatchedFrom":"172.23.97.100:37806","lastDispatchedTo":"172.23.120.162:11210","requestId":2332,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001999","opaque":"0xe55","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":12501,"encodingMicros":5,"totalMicros":12631,"serverMicros":0}} | +--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:41:12,463 | test | ERROR | MainProcess | pool-2-thread-1 | [task:call:1483] Failed to load 20 docs from 1875 to 2000 2020-01-13 22:41:12,579 | test | INFO | MainProcess | pool-2-thread-6 | [table_view:display:72] Ops Trend for bucket 'default' +---------+---------------------------------------------------------------------------------------------------------+---------+ | Min | Trend | Max | +---------+---------------------------------------------------------------------------------------------------------+---------+ | 0.000 | **....................................................................................................X | 330.000 | | 142.000 | ..........................................X | 142.000 | +---------+---------------------------------------------------------------------------------------------------------+---------+ 2020-01-13 22:41:34,747 | test | INFO | MainProcess | MainThread | [rest_client:monitorRebalance:1423] Rebalance done. Taken 25.0460000038 seconds to complete 2020-01-13 22:41:34,749 | test | INFO | MainProcess | MainThread | [rest_client:monitorRebalance:1425] Sleep for 5 seconds after rebalance 2020-01-13 22:42:41,069 | test | INFO | MainProcess | MainThread | [rest_client:monitorRebalance:1423] Rebalance done. Taken 55.0980000496 seconds to complete 2020-01-13 22:42:41,071 | test | INFO | MainProcess | MainThread | [rest_client:monitorRebalance:1425] Sleep for 5 seconds after rebalance 2020-01-13 22:42:46,295 | test | WARNING | MainProcess | MainThread | [sdk_client3:insert:476] The document already exists! => com.couchbase.client.core.error.DocumentExistsException: Document with the given id already exists {"completed":true,"coreId":18,"idempotent":false,"lastChannelId":"0000000000000012/000000002BD0F00F","lastDispatchedFrom":"172.23.97.100:60122","lastDispatchedTo":"172.23.120.192:11210","requestId":2405,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001068","opaque":"0xeca","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"EXISTS","timeoutMs":5000,"timings":{"dispatchMicros":301,"encodingMicros":11,"totalMicros":537,"serverMicros":0}} 2020-01-13 22:42:46,444 | test | INFO | MainProcess | MainThread | [swaprebalancetests:validate_docs:116] Validating docs 2020-01-13 22:42:46,944 | test | INFO | MainProcess | MainThread | [table_view:display:72] Cluster statistics +----------------+-----------------+-----------------+------------+------------+---------------------+-------------------+-----------------------+ | Node | Services | CPU_utilization | Mem_total | Mem_free | Swap_mem_used | Active / Replica | Version | +----------------+-----------------+-----------------+------------+------------+---------------------+-------------------+-----------------------+ | 172.23.120.192 | kv | 1.80412371134 | 3965001728 | 3192381440 | 0 / 3758092288 | 459 / 930 | 6.5.0-4960-enterprise | | 172.23.120.19 | kv | 3.0303030303 | 4220141568 | 3508654080 | 262144 / 3758092288 | 446 / 934 | 6.5.0-4960-enterprise | | 172.23.120.162 | index, kv, n1ql | 1.50753768844 | 3963588608 | 2995363840 | 0 / 2147479552 | 488 / 938 | 6.5.0-4960-enterprise | | 172.23.120.177 | kv | 1.5037593985 | 3965001728 | 3098202112 | 0 / 3758092288 | 483 / 950 | 6.5.0-4960-enterprise | +----------------+-----------------+-----------------+------------+------------+---------------------+-------------------+-----------------------+ 2020-01-13 22:42:46,960 | test | CRITICAL | MainProcess | MainThread | [basetestcase:tearDownEverything:344] Skipping get_trace !! 2020-01-13 22:43:13,786 | test | INFO | MainProcess | MainThread | [rest_client:monitorRebalance:1423] Rebalance done. Taken 25.0390000343 seconds to complete 2020-01-13 22:43:13,786 | test | INFO | MainProcess | MainThread | [rest_client:monitorRebalance:1425] Sleep for 5 seconds after rebalance 2020-01-13 22:43:49,584 | test | INFO | MainProcess | MainThread | [basetestcase:_BaseTestCase__log_setup_status:385] ========= basetestcase setup started for test #1 test_add_back_failed_node ========= 2020-01-13 22:43:50,128 | test | DEBUG | MainProcess | MainThread | [rest_client:is_ns_server_running:44] Checking if ns_server is running with timeout 360secs 2020-01-13 22:43:50,160 | test | DEBUG | MainProcess | MainThread | [rest_client:get_nodes_version:1681] Node version in cluster: 6.5.0-4960-enterprise 2020-01-13 22:43:50,161 | test | INFO | MainProcess | MainThread | [basetestcase:_BaseTestCase__log_setup_status:385] ========= basetestcase setup started for test #1 test_add_back_failed_node ========= 2020-01-13 22:43:50,161 | test | DEBUG | MainProcess | MainThread | [basetestcase:setUp:224] Cleaning up cluster 2020-01-13 22:43:50,229 | test | DEBUG | MainProcess | MainThread | [bucket_ready_functions:delete_all_buckets:246] Deleting existing buckets [] on 172.23.120.162 2020-01-13 22:43:50,263 | test | DEBUG | MainProcess | MainThread | [bucket_ready_functions:delete_all_buckets:246] Deleting existing buckets [] on 172.23.120.177 2020-01-13 22:43:50,299 | test | DEBUG | MainProcess | MainThread | [bucket_ready_functions:delete_all_buckets:246] Deleting existing buckets [] on 172.23.120.19 2020-01-13 22:43:50,325 | test | DEBUG | MainProcess | MainThread | [bucket_ready_functions:delete_all_buckets:246] Deleting existing buckets [] on 172.23.120.192 2020-01-13 22:43:50,354 | test | DEBUG | MainProcess | MainThread | [bucket_ready_functions:delete_all_buckets:246] Deleting existing buckets [] on 172.23.121.17 2020-01-13 22:43:50,381 | test | DEBUG | MainProcess | MainThread | [bucket_ready_functions:delete_all_buckets:246] Deleting existing buckets [] on 172.23.121.19 2020-01-13 22:43:50,433 | test | DEBUG | MainProcess | MainThread | [rest_client:is_ns_server_running:44] Checking if ns_server is running with timeout 120secs 2020-01-13 22:43:50,482 | test | DEBUG | MainProcess | MainThread | [cluster_ready_functions:wait_for_ns_servers_or_assert:109] Waiting for ns_server @ 172.23.120.162:8091 2020-01-13 22:43:50,483 | test | DEBUG | MainProcess | MainThread | [rest_client:is_ns_server_running:44] Checking if ns_server is running with timeout 360secs 2020-01-13 22:43:50,499 | test | DEBUG | MainProcess | MainThread | [cluster_ready_functions:wait_for_ns_servers_or_assert:112] ns_server @ 172.23.120.162:8091 is running 2020-01-13 22:43:50,538 | test | ERROR | MainProcess | MainThread | [rest_client:_http_request:737] GET http://172.23.120.162:8091/pools/default/ body: headers: {'Accept': '*/*', 'Connection': 'close', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n', 'Content-Type': 'application/x-www-form-urlencoded'} error: 404 reason: unknown "unknown pool" auth: Administrator:password 2020-01-13 22:43:50,540 | test | DEBUG | MainProcess | MainThread | [rest_client:_http_request:738] File "testrunner.py", line 514, in main() File "testrunner.py", line 428, in main result = unittest.TextTestRunner(verbosity=2).run(suite) File "/opt/jython/Lib/unittest/runner.py", line 151, in run test(result) File "/opt/jython/Lib/unittest/suite.py", line 70, in __call__ return self.run(*args, **kwds) File "/opt/jython/Lib/unittest/suite.py", line 108, in run test(result) File "/opt/jython/Lib/unittest/case.py", line 396, in __call__ return self.run(*args, **kwds) File "/opt/jython/Lib/unittest/case.py", line 323, in run self.setUp() File "pytests/rebalance_new/swaprebalancetests.py", line 559, in setUp super(SwapRebalanceFailedTests, self).setUp() File "pytests/rebalance_new/swaprebalancetests.py", line 20, in setUp super(SwapRebalanceBase, self).setUp() File "pytests/rebalance_new/rebalance_base.py", line 18, in setUp super(RebalanceBaseTest, self).setUp() File "pytests/basetestcase.py", line 247, in setUp self.tearDownEverything() File "pytests/basetestcase.py", line 347, in tearDownEverything alerts = rest.get_alerts() File "lib/membase/api/rest_client.py", line 2461, in get_alerts status, content, header = self._http_request(api) File "lib/membase/api/rest_client.py", line 738, in _http_request self.test_log.debug(''.join(traceback.format_stack())) 2020-01-13 22:43:50,540 | test | DEBUG | MainProcess | MainThread | [basetestcase:tearDownEverything:350] Cleaning up cluster 2020-01-13 22:43:50,588 | test | DEBUG | MainProcess | MainThread | [bucket_ready_functions:delete_all_buckets:246] Deleting existing buckets [] on 172.23.120.162 2020-01-13 22:43:50,615 | test | DEBUG | MainProcess | MainThread | [bucket_ready_functions:delete_all_buckets:246] Deleting existing buckets [] on 172.23.120.177 2020-01-13 22:43:50,641 | test | DEBUG | MainProcess | MainThread | [bucket_ready_functions:delete_all_buckets:246] Deleting existing buckets [] on 172.23.120.19 2020-01-13 22:43:50,661 | test | DEBUG | MainProcess | MainThread | [bucket_ready_functions:delete_all_buckets:246] Deleting existing buckets [] on 172.23.120.192 2020-01-13 22:43:50,773 | test | DEBUG | MainProcess | MainThread | [bucket_ready_functions:delete_all_buckets:246] Deleting existing buckets [] on 172.23.121.17 2020-01-13 22:43:50,792 | test | DEBUG | MainProcess | MainThread | [bucket_ready_functions:delete_all_buckets:246] Deleting existing buckets [] on 172.23.121.19 2020-01-13 22:43:50,832 | test | DEBUG | MainProcess | MainThread | [rest_client:is_ns_server_running:44] Checking if ns_server is running with timeout 120secs 2020-01-13 22:43:50,871 | test | DEBUG | MainProcess | MainThread | [cluster_ready_functions:wait_for_ns_servers_or_assert:109] Waiting for ns_server @ 172.23.120.162:8091 2020-01-13 22:43:50,871 | test | DEBUG | MainProcess | MainThread | [rest_client:is_ns_server_running:44] Checking if ns_server is running with timeout 360secs 2020-01-13 22:43:50,882 | test | DEBUG | MainProcess | MainThread | [cluster_ready_functions:wait_for_ns_servers_or_assert:112] ns_server @ 172.23.120.162:8091 is running 2020-01-13 22:43:53,657 | test | DEBUG | MainProcess | MainThread | [cluster_ready_functions:stop_server:360] Couchbase stopped on ip:172.23.120.162 port:8091 ssh_username:root 2020-01-13 22:43:54,282 | test | DEBUG | MainProcess | MainThread | [cluster_ready_functions:start_server:374] Couchbase started on ip:172.23.120.162 port:8091 ssh_username:root 2020-01-13 22:43:56,884 | test | DEBUG | MainProcess | MainThread | [cluster_ready_functions:stop_server:360] Couchbase stopped on ip:172.23.120.177 port:8091 ssh_username:root 2020-01-13 22:43:57,526 | test | DEBUG | MainProcess | MainThread | [cluster_ready_functions:start_server:374] Couchbase started on ip:172.23.120.177 port:8091 ssh_username:root 2020-01-13 22:44:01,463 | test | DEBUG | MainProcess | MainThread | [cluster_ready_functions:stop_server:360] Couchbase stopped on ip:172.23.120.19 port:8091 ssh_username:root 2020-01-13 22:44:02,538 | test | DEBUG | MainProcess | MainThread | [cluster_ready_functions:start_server:374] Couchbase started on ip:172.23.120.19 port:8091 ssh_username:root 2020-01-13 22:44:05,148 | test | DEBUG | MainProcess | MainThread | [cluster_ready_functions:stop_server:360] Couchbase stopped on ip:172.23.120.192 port:8091 ssh_username:root 2020-01-13 22:44:05,798 | test | DEBUG | MainProcess | MainThread | [cluster_ready_functions:start_server:374] Couchbase started on ip:172.23.120.192 port:8091 ssh_username:root 2020-01-13 22:44:08,272 | test | DEBUG | MainProcess | MainThread | [cluster_ready_functions:stop_server:360] Couchbase stopped on ip:172.23.121.17 port:8091 ssh_username:root 2020-01-13 22:44:08,760 | test | DEBUG | MainProcess | MainThread | [cluster_ready_functions:start_server:374] Couchbase started on ip:172.23.121.17 port:8091 ssh_username:root 2020-01-13 22:44:11,253 | test | DEBUG | MainProcess | MainThread | [cluster_ready_functions:stop_server:360] Couchbase stopped on ip:172.23.121.19 port:8091 ssh_username:root 2020-01-13 22:44:11,750 | test | DEBUG | MainProcess | MainThread | [cluster_ready_functions:start_server:374] Couchbase started on ip:172.23.121.19 port:8091 ssh_username:root 2020-01-13 22:44:21,750 | test | INFO | MainProcess | MainThread | [basetestcase:setUp:251] Initializing cluster 2020-01-13 22:44:21,802 | test | DEBUG | MainProcess | pool-4-thread-4 | [tasks:execute:155] server: ip:172.23.120.192 port:8091 ssh_username:root, nodes/self: {'memcached': 11210, 'memoryFree': 3396784128L, 'eventingMemoryQuota': 256, 'rest_password': '', 'storage': [], 'ports': [], 'cbasMemoryQuota': 1166, 'hostname': u'172.23.120.192:8091', 'ftsMemoryQuota': 256, 'clusterCompatibility': 393221, 'id': u'ns_1@172.23.120.192', 'mcdMemoryReserved': 3025, 'moxi': 11211, 'rest_username': '', 'os': u'x86_64-unknown-linux-gnu', 'ip': u'172.23.120.192', 'curr_items': 0, 'mcdMemoryAllocated': 3025, 'services': [u'kv'], 'version': u'6.5.0-4960-enterprise', 'indexMemoryQuota': 512, 'uptime': u'12', 'clusterMembership': u'active', 'storageTotalRam': 3781L, 'port': u'8091', 'availableStorage': [], 'memoryTotal': 3965001728L, 'memoryQuota': 1603, 'status': u'healthy'} 2020-01-13 22:44:21,802 | test | DEBUG | MainProcess | pool-4-thread-1 | [tasks:execute:155] server: ip:172.23.120.162 port:8091 ssh_username:root, nodes/self: {'memcached': 11210, 'memoryFree': 3277582336L, 'eventingMemoryQuota': 256, 'rest_password': '', 'storage': [], 'ports': [], 'cbasMemoryQuota': 1166, 'hostname': u'172.23.120.162:8091', 'ftsMemoryQuota': 256, 'clusterCompatibility': 393221, 'id': u'ns_1@172.23.120.162', 'mcdMemoryReserved': 3023, 'moxi': 11211, 'rest_username': '', 'os': u'x86_64-unknown-linux-gnu', 'ip': u'172.23.120.162', 'curr_items': 0, 'mcdMemoryAllocated': 3023, 'services': [u'kv'], 'version': u'6.5.0-4960-enterprise', 'indexMemoryQuota': 512, 'uptime': u'23', 'clusterMembership': u'active', 'storageTotalRam': 3779L, 'port': u'8091', 'availableStorage': [], 'memoryTotal': 3963588608L, 'memoryQuota': 1601, 'status': u'healthy'} 2020-01-13 22:44:21,803 | test | DEBUG | MainProcess | pool-4-thread-4 | [rest_client:init_cluster_memoryQuota:853] pools/default params: memoryQuota=2016 2020-01-13 22:44:21,802 | test | DEBUG | MainProcess | pool-4-thread-5 | [tasks:execute:155] server: ip:172.23.121.17 port:8091 ssh_username:root, nodes/self: {'memcached': 11210, 'memoryFree': 3015045120L, 'eventingMemoryQuota': 256, 'rest_password': '', 'storage': [], 'ports': [], 'cbasMemoryQuota': 1117, 'hostname': u'172.23.121.17:8091', 'ftsMemoryQuota': 256, 'clusterCompatibility': 393221, 'id': u'ns_1@cb.local', 'mcdMemoryReserved': 2827, 'moxi': 11211, 'rest_username': '', 'os': u'x86_64-unknown-linux-gnu', 'ip': u'172.23.121.17', 'curr_items': 0, 'mcdMemoryAllocated': 2827, 'services': [u'kv'], 'version': u'6.5.0-4960-enterprise', 'indexMemoryQuota': 512, 'uptime': u'7', 'clusterMembership': u'active', 'storageTotalRam': 3534L, 'port': u'8091', 'availableStorage': [], 'memoryTotal': 3705872384L, 'memoryQuota': 1454, 'status': u'healthy'} 2020-01-13 22:44:21,805 | test | DEBUG | MainProcess | pool-4-thread-1 | [tasks:execute:191] Quota for index service will be 256 MB 2020-01-13 22:44:21,805 | test | DEBUG | MainProcess | pool-4-thread-5 | [rest_client:init_cluster_memoryQuota:853] pools/default params: memoryQuota=1884 2020-01-13 22:44:21,805 | test | DEBUG | MainProcess | pool-4-thread-1 | [tasks:execute:194] Set index quota to node 172.23.120.162 2020-01-13 22:44:21,805 | test | DEBUG | MainProcess | pool-4-thread-1 | [rest_client:set_service_memoryQuota:865] pools/default params: indexMemoryQuota=256 2020-01-13 22:44:21,808 | test | DEBUG | MainProcess | pool-4-thread-3 | [tasks:execute:155] server: ip:172.23.120.19 port:8091 ssh_username:root, nodes/self: {'memcached': 11210, 'memoryFree': 3701243904L, 'eventingMemoryQuota': 256, 'rest_password': '', 'storage': [], 'ports': [], 'cbasMemoryQuota': 1215, 'hostname': u'172.23.120.19:8091', 'ftsMemoryQuota': 256, 'clusterCompatibility': 393221, 'id': u'ns_1@172.23.120.19', 'mcdMemoryReserved': 3219, 'moxi': 11211, 'rest_username': '', 'os': u'x86_64-unknown-linux-gnu', 'ip': u'172.23.120.19', 'curr_items': 0, 'mcdMemoryAllocated': 3219, 'services': [u'kv'], 'version': u'6.5.0-4960-enterprise', 'indexMemoryQuota': 512, 'uptime': u'12', 'clusterMembership': u'active', 'storageTotalRam': 4024L, 'port': u'8091', 'availableStorage': [], 'memoryTotal': 4220141568L, 'memoryQuota': 1748, 'status': u'healthy'} 2020-01-13 22:44:21,809 | test | DEBUG | MainProcess | pool-4-thread-3 | [rest_client:init_cluster_memoryQuota:853] pools/default params: memoryQuota=2146 2020-01-13 22:44:21,809 | test | DEBUG | MainProcess | pool-4-thread-2 | [tasks:execute:155] server: ip:172.23.120.177 port:8091 ssh_username:root, nodes/self: {'memcached': 11210, 'memoryFree': 3358093312L, 'eventingMemoryQuota': 256, 'rest_password': '', 'storage': [], 'ports': [], 'cbasMemoryQuota': 1166, 'hostname': u'172.23.120.177:8091', 'ftsMemoryQuota': 256, 'clusterCompatibility': 393221, 'id': u'ns_1@172.23.120.177', 'mcdMemoryReserved': 3025, 'moxi': 11211, 'rest_username': '', 'os': u'x86_64-unknown-linux-gnu', 'ip': u'172.23.120.177', 'curr_items': 0, 'mcdMemoryAllocated': 3025, 'services': [u'kv'], 'version': u'6.5.0-4960-enterprise', 'indexMemoryQuota': 512, 'uptime': u'18', 'clusterMembership': u'active', 'storageTotalRam': 3781L, 'port': u'8091', 'availableStorage': [], 'memoryTotal': 3965001728L, 'memoryQuota': 1603, 'status': u'healthy'} 2020-01-13 22:44:21,811 | test | DEBUG | MainProcess | pool-4-thread-6 | [tasks:execute:155] server: ip:172.23.121.19 port:8091 ssh_username:root, nodes/self: {'memcached': 11210, 'memoryFree': 909934592, 'eventingMemoryQuota': 256, 'rest_password': '', 'storage': [], 'ports': [], 'cbasMemoryQuota': 1117, 'hostname': u'172.23.121.19:8091', 'ftsMemoryQuota': 256, 'clusterCompatibility': 393221, 'id': u'ns_1@cb.local', 'mcdMemoryReserved': 2827, 'moxi': 11211, 'rest_username': '', 'os': u'x86_64-unknown-linux-gnu', 'ip': u'172.23.121.19', 'curr_items': 0, 'mcdMemoryAllocated': 2827, 'services': [u'kv'], 'version': u'6.5.0-4960-enterprise', 'indexMemoryQuota': 512, 'uptime': u'7', 'clusterMembership': u'active', 'storageTotalRam': 3534L, 'port': u'8091', 'availableStorage': [], 'memoryTotal': 3705872384L, 'memoryQuota': 1454, 'status': u'healthy'} 2020-01-13 22:44:21,811 | test | DEBUG | MainProcess | pool-4-thread-2 | [rest_client:init_cluster_memoryQuota:853] pools/default params: memoryQuota=2016 2020-01-13 22:44:21,812 | test | DEBUG | MainProcess | pool-4-thread-6 | [rest_client:init_cluster_memoryQuota:853] pools/default params: memoryQuota=1884 2020-01-13 22:44:21,818 | test | DEBUG | MainProcess | pool-4-thread-4 | [rest_client:set_indexer_storage_mode:887] settings/indexes params: storageMode=plasma 2020-01-13 22:44:21,819 | test | DEBUG | MainProcess | pool-4-thread-5 | [rest_client:set_indexer_storage_mode:887] settings/indexes params: storageMode=plasma 2020-01-13 22:44:21,822 | test | DEBUG | MainProcess | pool-4-thread-1 | [rest_client:init_cluster_memoryQuota:853] pools/default params: memoryQuota=1759 2020-01-13 22:44:21,825 | test | DEBUG | MainProcess | pool-4-thread-2 | [rest_client:set_indexer_storage_mode:887] settings/indexes params: storageMode=plasma 2020-01-13 22:44:21,828 | test | DEBUG | MainProcess | pool-4-thread-3 | [rest_client:set_indexer_storage_mode:887] settings/indexes params: storageMode=plasma 2020-01-13 22:44:21,831 | test | DEBUG | MainProcess | pool-4-thread-6 | [rest_client:set_indexer_storage_mode:887] settings/indexes params: storageMode=plasma 2020-01-13 22:44:21,834 | test | DEBUG | MainProcess | pool-4-thread-4 | [rest_client:init_cluster:758] settings/web params on 172.23.120.192:8091:password=password&port=8091&username=Administrator 2020-01-13 22:44:21,835 | test | DEBUG | MainProcess | pool-4-thread-5 | [rest_client:init_cluster:758] settings/web params on 172.23.121.17:8091:password=password&port=8091&username=Administrator 2020-01-13 22:44:21,838 | test | DEBUG | MainProcess | pool-4-thread-1 | [rest_client:set_indexer_storage_mode:887] settings/indexes params: storageMode=plasma 2020-01-13 22:44:21,841 | test | DEBUG | MainProcess | pool-4-thread-2 | [rest_client:init_cluster:758] settings/web params on 172.23.120.177:8091:password=password&port=8091&username=Administrator 2020-01-13 22:44:21,848 | test | DEBUG | MainProcess | pool-4-thread-6 | [rest_client:init_cluster:758] settings/web params on 172.23.121.19:8091:password=password&port=8091&username=Administrator 2020-01-13 22:44:21,849 | test | DEBUG | MainProcess | pool-4-thread-3 | [rest_client:init_cluster:758] settings/web params on 172.23.120.19:8091:password=password&port=8091&username=Administrator 2020-01-13 22:44:21,852 | test | DEBUG | MainProcess | pool-4-thread-1 | [rest_client:init_node_services:827] /node/controller/setupServices params on 172.23.120.162: 8091:services=kv%2Cn1ql%2Cindex&hostname=172.23.120.162&password=password&user=Administrator 2020-01-13 22:44:21,865 | test | DEBUG | MainProcess | pool-4-thread-1 | [rest_client:init_cluster:758] settings/web params on 172.23.120.162:8091:password=password&port=8091&username=Administrator 2020-01-13 22:44:21,930 | test | WARNING | MainProcess | MainThread | [basetestcase:_initialize_nodes:416] RAM quota was defined less than 100 MB: 2020-01-13 22:44:22,505 | test | DEBUG | MainProcess | MainThread | [basetestcase:_initialize_nodes:420] 172.23.120.162: ['MemTotal: 3870692 kB\n', 'MemFree: 910228 kB\n', 'MemAvailable: 2867432 kB\n', 'Buffers: 216 kB\n', 'Cached: 2259084 kB\n', 'SwapCached: 0 kB\n', 'Active: 1290128 kB\n', 'Inactive: 1364312 kB\n', 'Active(anon): 325988 kB\n', 'Inactive(anon): 248080 kB\n', 'Active(file): 964140 kB\n', 'Inactive(file): 1116232 kB\n', 'Unevictable: 0 kB\n', 'Mlocked: 0 kB\n', 'SwapTotal: 2097148 kB\n', 'SwapFree: 2097148 kB\n', 'Dirty: 1180 kB\n', 'Writeback: 0 kB\n', 'AnonPages: 395188 kB\n', 'Mapped: 80464 kB\n', 'Shmem: 178924 kB\n', 'Slab: 224340 kB\n', 'SReclaimable: 156884 kB\n', 'SUnreclaim: 67456 kB\n', 'KernelStack: 5328 kB\n', 'PageTables: 9472 kB\n', 'NFS_Unstable: 0 kB\n', 'Bounce: 0 kB\n', 'WritebackTmp: 0 kB\n', 'CommitLimit: 4032492 kB\n', 'Committed_AS: 2193384 kB\n', 'VmallocTotal: 34359738367 kB\n', 'VmallocUsed: 25228 kB\n', 'VmallocChunk: 34359707644 kB\n', 'HardwareCorrupted: 0 kB\n', 'AnonHugePages: 4096 kB\n', 'CmaTotal: 0 kB\n', 'CmaFree: 0 kB\n', 'HugePages_Total: 0\n', 'HugePages_Free: 0\n', 'HugePages_Rsvd: 0\n', 'HugePages_Surp: 0\n', 'Hugepagesize: 2048 kB\n', 'DirectMap4k: 129024 kB\n', 'DirectMap2M: 4057088 kB\n', 'DirectMap1G: 0 kB\n'] 2020-01-13 22:44:23,019 | test | DEBUG | MainProcess | MainThread | [basetestcase:_initialize_nodes:420] 172.23.120.177: ['MemTotal: 3872072 kB\n', 'MemFree: 877912 kB\n', 'MemAvailable: 2947592 kB\n', 'Buffers: 188 kB\n', 'Cached: 2392332 kB\n', 'SwapCached: 0 kB\n', 'Active: 1218280 kB\n', 'Inactive: 1405196 kB\n', 'Active(anon): 205656 kB\n', 'Inactive(anon): 254452 kB\n', 'Active(file): 1012624 kB\n', 'Inactive(file): 1150744 kB\n', 'Unevictable: 7740 kB\n', 'Mlocked: 7740 kB\n', 'SwapTotal: 3670012 kB\n', 'SwapFree: 3670012 kB\n', 'Dirty: 892 kB\n', 'Writeback: 0 kB\n', 'AnonPages: 238544 kB\n', 'Mapped: 49660 kB\n', 'Shmem: 225548 kB\n', 'Slab: 258168 kB\n', 'SReclaimable: 192404 kB\n', 'SUnreclaim: 65764 kB\n', 'KernelStack: 5600 kB\n', 'PageTables: 6744 kB\n', 'NFS_Unstable: 0 kB\n', 'Bounce: 0 kB\n', 'WritebackTmp: 0 kB\n', 'CommitLimit: 5606048 kB\n', 'Committed_AS: 1531668 kB\n', 'VmallocTotal: 34359738367 kB\n', 'VmallocUsed: 26312 kB\n', 'VmallocChunk: 34359707644 kB\n', 'HardwareCorrupted: 0 kB\n', 'AnonHugePages: 10240 kB\n', 'HugePages_Total: 0\n', 'HugePages_Free: 0\n', 'HugePages_Rsvd: 0\n', 'HugePages_Surp: 0\n', 'Hugepagesize: 2048 kB\n', 'DirectMap4k: 112640 kB\n', 'DirectMap2M: 4073472 kB\n', 'DirectMap1G: 0 kB\n'] 2020-01-13 22:44:23,857 | test | DEBUG | MainProcess | MainThread | [basetestcase:_initialize_nodes:420] 172.23.120.19: ['MemTotal: 4121232 kB\n', 'MemFree: 859740 kB\n', 'MemAvailable: 3267480 kB\n', 'Buffers: 61864 kB\n', 'Cached: 2687360 kB\n', 'SwapCached: 0 kB\n', 'Active: 1295232 kB\n', 'Inactive: 1671400 kB\n', 'Active(anon): 191336 kB\n', 'Inactive(anon): 238108 kB\n', 'Active(file): 1103896 kB\n', 'Inactive(file): 1433292 kB\n', 'Unevictable: 8368 kB\n', 'Mlocked: 8368 kB\n', 'SwapTotal: 3670012 kB\n', 'SwapFree: 3669756 kB\n', 'Dirty: 924 kB\n', 'Writeback: 0 kB\n', 'AnonPages: 225808 kB\n', 'Mapped: 53764 kB\n', 'Shmem: 208396 kB\n', 'Slab: 201228 kB\n', 'SReclaimable: 147188 kB\n', 'SUnreclaim: 54040 kB\n', 'KernelStack: 5648 kB\n', 'PageTables: 6804 kB\n', 'NFS_Unstable: 0 kB\n', 'Bounce: 0 kB\n', 'WritebackTmp: 0 kB\n', 'CommitLimit: 5730628 kB\n', 'Committed_AS: 1497416 kB\n', 'VmallocTotal: 34359738367 kB\n', 'VmallocUsed: 33412 kB\n', 'VmallocChunk: 34359696380 kB\n', 'HardwareCorrupted: 0 kB\n', 'AnonHugePages: 4096 kB\n', 'CmaTotal: 0 kB\n', 'CmaFree: 0 kB\n', 'HugePages_Total: 0\n', 'HugePages_Free: 0\n', 'HugePages_Rsvd: 0\n', 'HugePages_Surp: 0\n', 'Hugepagesize: 2048 kB\n', 'DirectMap4k: 114688 kB\n', 'DirectMap2M: 4325376 kB\n'] 2020-01-13 22:44:24,354 | test | DEBUG | MainProcess | MainThread | [basetestcase:_initialize_nodes:420] 172.23.120.192: ['MemTotal: 3872072 kB\n', 'MemFree: 762060 kB\n', 'MemAvailable: 2952748 kB\n', 'Buffers: 212 kB\n', 'Cached: 2547540 kB\n', 'SwapCached: 0 kB\n', 'Active: 1389516 kB\n', 'Inactive: 1427384 kB\n', 'Active(anon): 194976 kB\n', 'Inactive(anon): 302944 kB\n', 'Active(file): 1194540 kB\n', 'Inactive(file): 1124440 kB\n', 'Unevictable: 7792 kB\n', 'Mlocked: 7792 kB\n', 'SwapTotal: 3670012 kB\n', 'SwapFree: 3670012 kB\n', 'Dirty: 1732 kB\n', 'Writeback: 0 kB\n', 'AnonPages: 276792 kB\n', 'Mapped: 54804 kB\n', 'Shmem: 225172 kB\n', 'Slab: 204036 kB\n', 'SReclaimable: 146652 kB\n', 'SUnreclaim: 57384 kB\n', 'KernelStack: 5936 kB\n', 'PageTables: 7548 kB\n', 'NFS_Unstable: 0 kB\n', 'Bounce: 0 kB\n', 'WritebackTmp: 0 kB\n', 'CommitLimit: 5606048 kB\n', 'Committed_AS: 1814708 kB\n', 'VmallocTotal: 34359738367 kB\n', 'VmallocUsed: 26744 kB\n', 'VmallocChunk: 34359707644 kB\n', 'HardwareCorrupted: 0 kB\n', 'AnonHugePages: 4096 kB\n', 'HugePages_Total: 0\n', 'HugePages_Free: 0\n', 'HugePages_Rsvd: 0\n', 'HugePages_Surp: 0\n', 'Hugepagesize: 2048 kB\n', 'DirectMap4k: 114688 kB\n', 'DirectMap2M: 4071424 kB\n', 'DirectMap1G: 0 kB\n'] 2020-01-13 22:44:24,733 | test | DEBUG | MainProcess | MainThread | [basetestcase:_initialize_nodes:420] 172.23.121.17: ['MemTotal: 3619016 kB\n', 'MemFree: 850060 kB\n', 'MemAvailable: 2666436 kB\n', 'Buffers: 120612 kB\n', 'Cached: 1983552 kB\n', 'SwapCached: 0 kB\n', 'Active: 1053892 kB\n', 'Inactive: 1434008 kB\n', 'Active(anon): 320852 kB\n', 'Inactive(anon): 252260 kB\n', 'Active(file): 733040 kB\n', 'Inactive(file): 1181748 kB\n', 'Unevictable: 7784 kB\n', 'Mlocked: 7784 kB\n', 'SwapTotal: 3670012 kB\n', 'SwapFree: 3670012 kB\n', 'Dirty: 1064 kB\n', 'Writeback: 0 kB\n', 'AnonPages: 391400 kB\n', 'Mapped: 53672 kB\n', 'Shmem: 185792 kB\n', 'Slab: 192960 kB\n', 'SReclaimable: 141060 kB\n', 'SUnreclaim: 51900 kB\n', 'KernelStack: 6128 kB\n', 'PageTables: 7448 kB\n', 'NFS_Unstable: 0 kB\n', 'Bounce: 0 kB\n', 'WritebackTmp: 0 kB\n', 'CommitLimit: 5479520 kB\n', 'Committed_AS: 1681220 kB\n', 'VmallocTotal: 34359738367 kB\n', 'VmallocUsed: 24164 kB\n', 'VmallocChunk: 34359707648 kB\n', 'HardwareCorrupted: 0 kB\n', 'AnonHugePages: 6144 kB\n', 'HugePages_Total: 0\n', 'HugePages_Free: 0\n', 'HugePages_Rsvd: 0\n', 'HugePages_Surp: 0\n', 'Hugepagesize: 2048 kB\n', 'DirectMap4k: 96256 kB\n', 'DirectMap2M: 4089856 kB\n', 'DirectMap1G: 0 kB\n'] 2020-01-13 22:44:25,099 | test | DEBUG | MainProcess | MainThread | [basetestcase:_initialize_nodes:420] 172.23.121.19: ['MemTotal: 3619016 kB\n', 'MemFree: 119380 kB\n', 'MemAvailable: 571252 kB\n', 'Buffers: 184 kB\n', 'Cached: 771400 kB\n', 'SwapCached: 166012 kB\n', 'Active: 2228420 kB\n', 'Inactive: 1037120 kB\n', 'Active(anon): 1937956 kB\n', 'Inactive(anon): 752912 kB\n', 'Active(file): 290464 kB\n', 'Inactive(file): 284208 kB\n', 'Unevictable: 7868 kB\n', 'Mlocked: 7868 kB\n', 'SwapTotal: 3670012 kB\n', 'SwapFree: 3122452 kB\n', 'Dirty: 1092 kB\n', 'Writeback: 0 kB\n', 'AnonPages: 2337372 kB\n', 'Mapped: 58752 kB\n', 'Shmem: 193324 kB\n', 'Slab: 144296 kB\n', 'SReclaimable: 92288 kB\n', 'SUnreclaim: 52008 kB\n', 'KernelStack: 7024 kB\n', 'PageTables: 12668 kB\n', 'NFS_Unstable: 0 kB\n', 'Bounce: 0 kB\n', 'WritebackTmp: 0 kB\n', 'CommitLimit: 5479520 kB\n', 'Committed_AS: 4174472 kB\n', 'VmallocTotal: 34359738367 kB\n', 'VmallocUsed: 24164 kB\n', 'VmallocChunk: 34359710716 kB\n', 'HardwareCorrupted: 0 kB\n', 'AnonHugePages: 1542144 kB\n', 'HugePages_Total: 0\n', 'HugePages_Free: 0\n', 'HugePages_Rsvd: 0\n', 'HugePages_Surp: 0\n', 'Hugepagesize: 2048 kB\n', 'DirectMap4k: 100352 kB\n', 'DirectMap2M: 3037184 kB\n', 'DirectMap1G: 1048576 kB\n'] 2020-01-13 22:44:25,101 | test | INFO | MainProcess | MainThread | [basetestcase:setUp:273] Couchbase Cluster: default, Nodes: 172.23.120.162, 172.23.120.177, 172.23.120.19, 172.23.120.192, 172.23.121.17, 172.23.121.19 initialized 2020-01-13 22:44:25,114 | test | INFO | MainProcess | MainThread | [basetestcase:_BaseTestCase__log_setup_status:385] ========= basetestcase setup finished for test #1 test_add_back_failed_node ========= 2020-01-13 22:44:25,145 | test | DEBUG | MainProcess | MainThread | [basetestcase:sleep:482] Sleep is called from pytests/basetestcase.py -> setUp():L294 2020-01-13 22:44:25,147 | test | INFO | MainProcess | MainThread | [basetestcase:sleep:486] Reason: None. Sleep for 5 secs ... 2020-01-13 22:44:30,168 | test | DEBUG | MainProcess | MainThread | [rest_client:init_cluster:758] settings/web params on 172.23.120.162:8091:password=password&port=8091&username=Administrator 2020-01-13 22:44:30,209 | test | DEBUG | MainProcess | MainThread | [rest_client:init_cluster_memoryQuota:853] pools/default params: memoryQuota=2015 2020-01-13 22:44:30,240 | test | DEBUG | MainProcess | pool-4-thread-7 | [rest_client:add_node:1093] Adding remote node 172.23.120.177:8091 to cluster 172.23.120.162:8091 2020-01-13 22:44:31,950 | test | DEBUG | MainProcess | pool-4-thread-7 | [rest_client:add_node:1093] Adding remote node 172.23.120.19:8091 to cluster 172.23.120.162:8091 2020-01-13 22:44:35,717 | test | DEBUG | MainProcess | pool-4-thread-7 | [rest_client:add_node:1093] Adding remote node 172.23.120.192:8091 to cluster 172.23.120.162:8091 2020-01-13 22:44:37,266 | test | DEBUG | MainProcess | pool-4-thread-7 | [rest_client:rebalance:1327] Rebalance params: {'password': 'password', 'knownNodes': u'ns_1@172.23.120.192,ns_1@172.23.120.19,ns_1@172.23.120.162,ns_1@172.23.120.177', 'ejectedNodes': '', 'user': 'Administrator'} 2020-01-13 22:44:37,279 | test | DEBUG | MainProcess | pool-4-thread-7 | [rest_client:rebalance:1332] Rebalance operation started 2020-01-13 22:44:37,280 | test | INFO | MainProcess | pool-4-thread-7 | [table_view:display:72] Rebalance Overview +----------------+-----------------+--------------+ | Nodes | Services | Status | +----------------+-----------------+--------------+ | 172.23.120.162 | index, kv, n1ql | Cluster node | | 172.23.120.177 | None | <--- IN --- | | 172.23.120.19 | None | <--- IN --- | | 172.23.120.192 | None | <--- IN --- | +----------------+-----------------+--------------+ 2020-01-13 22:44:37,289 | test | DEBUG | MainProcess | pool-4-thread-7 | [rest_client:_rebalance_status_and_progress:1486] Rebalance percentage: 0.00 % 2020-01-13 22:44:37,290 | test | INFO | MainProcess | pool-4-thread-7 | [task:check:317] Rebalance - status: running, progress: 0 2020-01-13 22:44:52,323 | test | INFO | MainProcess | pool-4-thread-7 | [task:check:317] Rebalance - status: none, progress: 100 2020-01-13 22:44:52,338 | test | INFO | MainProcess | pool-4-thread-7 | [task:check:376] Rebalance completed with progress: 100% in 15.0590000153 sec 2020-01-13 22:44:52,339 | test | DEBUG | MainProcess | MainThread | [bucket_ready_functions:add_rbac_user:2292] **** Add built-in 'cbadminbucket' user to node 172.23.120.162 **** 2020-01-13 22:44:52,411 | test | DEBUG | MainProcess | MainThread | [bucket_ready_functions:add_rbac_user:2296] **** Add 'admin' role to 'cbadminbucket' user **** 2020-01-13 22:44:52,470 | test | DEBUG | MainProcess | MainThread | [bucket_ready_functions:async_create_bucket:141] Creating bucket: default 2020-01-13 22:44:52,526 | test | DEBUG | MainProcess | MainThread | [bucket_ready_functions:sleep:85] Sleep for 2 secs. ... 2020-01-13 22:44:54,578 | test | DEBUG | MainProcess | MainThread | [cluster_ready_functions:get_nodes_from_services_map:451] All nodes in cluster: [ip:172.23.120.192 port:8091 ssh_username:root, ip:172.23.120.19 port:8091 ssh_username:root, ip:172.23.120.162 port:8091 ssh_username:root, ip:172.23.120.177 port:8091 ssh_username:root] 2020-01-13 22:44:54,578 | test | DEBUG | MainProcess | MainThread | [bucket_ready_functions:_wait_warmup_completed:2240] Waiting for bucket default to complete warmup 2020-01-13 22:44:57,678 | test | DEBUG | MainProcess | MainThread | [bucket_ready_functions:parse_get_bucket_json:2064] read 1024 vbuckets 2020-01-13 22:44:57,680 | test | DEBUG | MainProcess | MainThread | [bucket_ready_functions:parse_get_bucket_json:2067] Stats: {u'opsPerSec': 32, u'diskFetches': 0, u'dataUsed': 36578691, u'itemCount': 500, u'quotaPercentUsed': 1.587025975435417, u'storageTotals': {u'hdd': {u'usedByData': 25712571, u'used': 38901033409L, u'quotaTotal': 144901136384L, u'total': 144901136384L, u'free': 75874132132L}, u'ram': {u'usedByData': 89396384, u'used': 11969437696L, u'quotaTotal': 8451522560L, u'total': 16113733632L, u'quotaUsed': 5632950272L, u'quotaUsedPerNode': 1408237568, u'quotaTotalPerNode': 2112880640}}, u'diskUsed': 25712571, u'vbActiveNumNonResident': 0, u'memUsed': 89396384} 2020-01-13 22:44:57,694 | test | DEBUG | MainProcess | MainThread | [sdk_client3:_createConn:96] Creating cluster connection 2020-01-13 22:44:57,709 | test | DEBUG | MainProcess | MainThread | [sdk_client3:_createConn:96] Creating cluster connection 2020-01-13 22:44:57,724 | test | DEBUG | MainProcess | MainThread | [sdk_client3:_createConn:96] Creating cluster connection 2020-01-13 22:44:57,744 | test | DEBUG | MainProcess | MainThread | [sdk_client3:_createConn:96] Creating cluster connection 2020-01-13 22:44:57,769 | test | DEBUG | MainProcess | MainThread | [sdk_client3:_createConn:96] Creating cluster connection 2020-01-13 22:44:57,786 | test | DEBUG | MainProcess | MainThread | [sdk_client3:_createConn:96] Creating cluster connection 2020-01-13 22:44:57,808 | test | DEBUG | MainProcess | MainThread | [sdk_client3:_createConn:96] Creating cluster connection 2020-01-13 22:44:57,842 | test | DEBUG | MainProcess | MainThread | [sdk_client3:_createConn:96] Creating cluster connection 2020-01-13 22:44:57,920 | test | DEBUG | MainProcess | pool-4-thread-10 | [task:call:2870] Ops rate for 'default': 0.000000 2020-01-13 22:44:58,092 | test | DEBUG | MainProcess | pool-4-thread-9 | [task:call:1483] Failed to load 0 docs from 0 to 125 2020-01-13 22:44:58,092 | test | DEBUG | MainProcess | pool-4-thread-9 | [task:call:1483] Failed to load 0 docs from 125 to 250 2020-01-13 22:44:58,092 | test | DEBUG | MainProcess | pool-4-thread-9 | [task:call:1483] Failed to load 0 docs from 250 to 375 2020-01-13 22:44:58,092 | test | DEBUG | MainProcess | pool-4-thread-9 | [task:call:1483] Failed to load 0 docs from 375 to 500 2020-01-13 22:44:58,094 | test | DEBUG | MainProcess | pool-4-thread-9 | [task:call:1483] Failed to load 0 docs from 500 to 625 2020-01-13 22:44:58,094 | test | DEBUG | MainProcess | pool-4-thread-9 | [task:call:1483] Failed to load 0 docs from 625 to 750 2020-01-13 22:44:58,095 | test | DEBUG | MainProcess | pool-4-thread-9 | [task:call:1483] Failed to load 0 docs from 750 to 875 2020-01-13 22:44:58,095 | test | DEBUG | MainProcess | pool-4-thread-9 | [task:call:1483] Failed to load 0 docs from 875 to 1000 2020-01-13 22:44:58,921 | test | DEBUG | MainProcess | pool-4-thread-9 | [sdk_client3:close:143] Closing down the cluster 2020-01-13 22:44:58,948 | test | DEBUG | MainProcess | pool-4-thread-9 | [sdk_client3:close:147] Cluster disconnected and env shutdown 2020-01-13 22:44:58,950 | test | DEBUG | MainProcess | pool-4-thread-9 | [sdk_client3:close:143] Closing down the cluster 2020-01-13 22:44:58,976 | test | DEBUG | MainProcess | pool-4-thread-9 | [sdk_client3:close:147] Cluster disconnected and env shutdown 2020-01-13 22:44:58,976 | test | DEBUG | MainProcess | pool-4-thread-9 | [sdk_client3:close:143] Closing down the cluster 2020-01-13 22:44:59,003 | test | DEBUG | MainProcess | pool-4-thread-9 | [sdk_client3:close:147] Cluster disconnected and env shutdown 2020-01-13 22:44:59,003 | test | DEBUG | MainProcess | pool-4-thread-9 | [sdk_client3:close:143] Closing down the cluster 2020-01-13 22:44:59,032 | test | DEBUG | MainProcess | pool-4-thread-9 | [sdk_client3:close:147] Cluster disconnected and env shutdown 2020-01-13 22:44:59,032 | test | DEBUG | MainProcess | pool-4-thread-9 | [sdk_client3:close:143] Closing down the cluster 2020-01-13 22:44:59,059 | test | DEBUG | MainProcess | pool-4-thread-9 | [sdk_client3:close:147] Cluster disconnected and env shutdown 2020-01-13 22:44:59,059 | test | DEBUG | MainProcess | pool-4-thread-9 | [sdk_client3:close:143] Closing down the cluster 2020-01-13 22:44:59,085 | test | DEBUG | MainProcess | pool-4-thread-9 | [sdk_client3:close:147] Cluster disconnected and env shutdown 2020-01-13 22:44:59,086 | test | DEBUG | MainProcess | pool-4-thread-9 | [sdk_client3:close:143] Closing down the cluster 2020-01-13 22:44:59,115 | test | DEBUG | MainProcess | pool-4-thread-9 | [sdk_client3:close:147] Cluster disconnected and env shutdown 2020-01-13 22:44:59,117 | test | DEBUG | MainProcess | pool-4-thread-9 | [sdk_client3:close:143] Closing down the cluster 2020-01-13 22:44:59,141 | test | DEBUG | MainProcess | pool-4-thread-9 | [sdk_client3:close:147] Cluster disconnected and env shutdown 2020-01-13 22:44:59,154 | test | DEBUG | MainProcess | MainThread | [sdk_client3:_createConn:96] Creating cluster connection 2020-01-13 22:44:59,161 | test | DEBUG | MainProcess | MainThread | [sdk_client3:close:143] Closing down the cluster 2020-01-13 22:44:59,186 | test | DEBUG | MainProcess | MainThread | [sdk_client3:close:147] Cluster disconnected and env shutdown 2020-01-13 22:44:59,187 | test | INFO | MainProcess | MainThread | [rebalance_base:setUp:65] Verifying num_items counts after doc_ops 2020-01-13 22:44:59,243 | test | DEBUG | MainProcess | MainThread | [cluster_ready_functions:get_nodes_from_services_map:451] All nodes in cluster: [ip:172.23.120.192 port:8091 ssh_username:root, ip:172.23.120.19 port:8091 ssh_username:root, ip:172.23.120.162 port:8091 ssh_username:root, ip:172.23.120.177 port:8091 ssh_username:root] 2020-01-13 22:44:59,842 | test | DEBUG | MainProcess | pool-4-thread-1 | [task:_compare:2192] Comparing 0 == 0 2020-01-13 22:44:59,842 | test | DEBUG | MainProcess | pool-4-thread-1 | [task:_get_all_stats_and_compare:2181] Ready: ep_queue_size 0 == 0. Received: {'172.23.120.162': '0'} for bucket 'default' 2020-01-13 22:45:00,361 | test | DEBUG | MainProcess | pool-4-thread-7 | [task:_compare:2192] Comparing 0 == 0 2020-01-13 22:45:00,361 | test | DEBUG | MainProcess | pool-4-thread-7 | [task:_get_all_stats_and_compare:2181] Ready: ep_queue_size 0 == 0. Received: {'172.23.120.177': '0'} for bucket 'default' 2020-01-13 22:45:01,421 | test | DEBUG | MainProcess | pool-4-thread-2 | [task:_compare:2192] Comparing 0 == 0 2020-01-13 22:45:01,421 | test | DEBUG | MainProcess | pool-4-thread-2 | [task:_get_all_stats_and_compare:2181] Ready: ep_queue_size 0 == 0. Received: {'172.23.120.19': '0'} for bucket 'default' 2020-01-13 22:45:01,773 | test | DEBUG | MainProcess | pool-4-thread-3 | [task:_compare:2192] Comparing 0 == 0 2020-01-13 22:45:01,773 | test | DEBUG | MainProcess | pool-4-thread-3 | [task:_get_all_stats_and_compare:2181] Ready: ep_queue_size 0 == 0. Received: {'172.23.120.192': '0'} for bucket 'default' 2020-01-13 22:45:01,828 | test | DEBUG | MainProcess | MainThread | [cluster_ready_functions:get_nodes_from_services_map:451] All nodes in cluster: [ip:172.23.120.192 port:8091 ssh_username:root, ip:172.23.120.19 port:8091 ssh_username:root, ip:172.23.120.162 port:8091 ssh_username:root, ip:172.23.120.177 port:8091 ssh_username:root] 2020-01-13 22:45:01,851 | test | INFO | MainProcess | MainThread | [data_helper:direct_client:268] creating direct client 172.23.120.162:11210 default 2020-01-13 22:45:04,108 | test | INFO | MainProcess | MainThread | [data_helper:direct_client:268] creating direct client 172.23.120.177:11210 default 2020-01-13 22:45:05,967 | test | INFO | MainProcess | MainThread | [data_helper:direct_client:268] creating direct client 172.23.120.19:11210 default 2020-01-13 22:45:07,881 | test | INFO | MainProcess | MainThread | [data_helper:direct_client:268] creating direct client 172.23.120.192:11210 default 2020-01-13 22:45:10,151 | test | DEBUG | MainProcess | MainThread | [bucket_ready_functions:verify_stats_for_bucket:519] Verifying stats for bucket default 2020-01-13 22:45:10,204 | test | DEBUG | MainProcess | MainThread | [cluster_ready_functions:get_nodes_from_services_map:451] All nodes in cluster: [ip:172.23.120.192 port:8091 ssh_username:root, ip:172.23.120.19 port:8091 ssh_username:root, ip:172.23.120.162 port:8091 ssh_username:root, ip:172.23.120.177 port:8091 ssh_username:root] 2020-01-13 22:45:14,960 | test | DEBUG | MainProcess | MainThread | [bucket_ready_functions:sleep:85] Sleep for 5 secs. ... 2020-01-13 22:45:15,605 | test | DEBUG | MainProcess | pool-4-thread-8 | [task:_compare:2192] Comparing 2000 == 2000 2020-01-13 22:45:15,605 | test | DEBUG | MainProcess | pool-4-thread-8 | [task:_get_all_stats_and_compare:2181] Ready: vb_replica_curr_items 2000 == 2000. Received: {'172.23.120.162': '504', '172.23.120.192': '491', '172.23.120.177': '501', '172.23.120.19': '504'} for bucket 'default' 2020-01-13 22:45:20,575 | test | DEBUG | MainProcess | pool-4-thread-4 | [task:_compare:2192] Comparing 3000 == 3000 2020-01-13 22:45:20,575 | test | DEBUG | MainProcess | pool-4-thread-4 | [task:_get_all_stats_and_compare:2181] Ready: curr_items_tot 3000 == 3000. Received: {'172.23.120.162': '754', '172.23.120.192': '741', '172.23.120.177': '751', '172.23.120.19': '754'} for bucket 'default' 2020-01-13 22:45:20,621 | test | INFO | MainProcess | MainThread | [table_view:display:72] Cluster statistics +----------------+-----------------+-----------------+------------+------------+---------------------+-------------------+-----------------------+ | Node | Services | CPU_utilization | Mem_total | Mem_free | Swap_mem_used | Active / Replica | Version | +----------------+-----------------+-----------------+------------+------------+---------------------+-------------------+-----------------------+ | 172.23.120.192 | kv | 4.84693877551 | 3965001728 | 3318874112 | 0 / 3758092288 | 250 / 491 | 6.5.0-4960-enterprise | | 172.23.120.19 | kv | 9.39086294416 | 4220141568 | 3609206784 | 262144 / 3758092288 | 250 / 504 | 6.5.0-4960-enterprise | | 172.23.120.162 | index, kv, n1ql | 1.50753768844 | 3963588608 | 3140636672 | 0 / 2147479552 | 250 / 504 | 6.5.0-4960-enterprise | | 172.23.120.177 | kv | 1.50753768844 | 3965001728 | 3280310272 | 0 / 3758092288 | 250 / 501 | 6.5.0-4960-enterprise | +----------------+-----------------+-----------------+------------+------------+---------------------+-------------------+-----------------------+ 2020-01-13 22:45:20,661 | test | DEBUG | MainProcess | MainThread | [bucket_ready_functions:parse_get_bucket_json:2064] read 1024 vbuckets 2020-01-13 22:45:20,663 | test | DEBUG | MainProcess | MainThread | [bucket_ready_functions:parse_get_bucket_json:2067] Stats: {u'opsPerSec': 0, u'diskFetches': 0, u'dataUsed': 28509920, u'itemCount': 1000, u'quotaPercentUsed': 1.605237675352231, u'storageTotals': {u'hdd': {u'usedByData': 28564945, u'used': 38901033409L, u'quotaTotal': 144901136384L, u'total': 144901136384L, u'free': 75874132132L}, u'ram': {u'usedByData': 90422240, u'used': 12922408960L, u'quotaTotal': 8451522560L, u'total': 16113733632L, u'quotaUsed': 5632950272L, u'quotaUsedPerNode': 1408237568, u'quotaTotalPerNode': 2112880640}}, u'diskUsed': 28564945, u'vbActiveNumNonResident': 0, u'memUsed': 90422240} 2020-01-13 22:45:20,663 | test | INFO | MainProcess | MainThread | [table_view:display:72] Bucket statistics +---------+---------+----------+-----+-------+------------+----------+-----------+ | Bucket | Type | Replicas | TTL | Items | RAM Quota | RAM Used | Disk Used | +---------+---------+----------+-----+-------+------------+----------+-----------+ | default | membase | 2 | 0 | 1000 | 5632950272 | 90422240 | 28564945 | +---------+---------+----------+-----+-------+------------+----------+-----------+ 2020-01-13 22:45:20,664 | test | INFO | MainProcess | MainThread | [rebalance_base:setUp:85] ==========Finished rebalance base setup======== 2020-01-13 22:45:20,664 | test | INFO | MainProcess | MainThread | [swaprebalancetests:setUp:21] === SwapRebalanceBase setup started for test #1 test_add_back_failed_node === 2020-01-13 22:45:20,676 | test | INFO | MainProcess | MainThread | [swaprebalancetests:setUp:44] picking server : ip:172.23.120.162 port:8091 ssh_username:root as the master 2020-01-13 22:45:21,210 | test | INFO | MainProcess | MainThread | [swaprebalancetests:enable_diag_eval_on_non_local_hosts:90] Enabled diag/eval for non-local hosts from 172.23.120.162 2020-01-13 22:45:21,232 | test | INFO | MainProcess | MainThread | [swaprebalancetests:setUp:69] === SwapRebalanceBase setup done for test #1 test_add_back_failed_node === 2020-01-13 22:45:21,270 | test | DEBUG | MainProcess | MainThread | [bucket_ready_functions:parse_get_bucket_json:2064] read 1024 vbuckets 2020-01-13 22:45:21,270 | test | DEBUG | MainProcess | MainThread | [bucket_ready_functions:parse_get_bucket_json:2067] Stats: {u'opsPerSec': 0, u'diskFetches': 0, u'dataUsed': 28509920, u'itemCount': 1000, u'quotaPercentUsed': 1.605237675352231, u'storageTotals': {u'hdd': {u'usedByData': 28564945, u'used': 38901033409L, u'quotaTotal': 144901136384L, u'total': 144901136384L, u'free': 75874132132L}, u'ram': {u'usedByData': 90422240, u'used': 12922408960L, u'quotaTotal': 8451522560L, u'total': 16113733632L, u'quotaUsed': 5632950272L, u'quotaUsedPerNode': 1408237568, u'quotaTotalPerNode': 2112880640}}, u'diskUsed': 28564945, u'vbActiveNumNonResident': 0, u'memUsed': 90422240} 2020-01-13 22:45:21,285 | test | DEBUG | MainProcess | MainThread | [sdk_client3:_createConn:96] Creating cluster connection 2020-01-13 22:45:21,301 | test | DEBUG | MainProcess | MainThread | [sdk_client3:_createConn:96] Creating cluster connection 2020-01-13 22:45:21,316 | test | DEBUG | MainProcess | MainThread | [sdk_client3:_createConn:96] Creating cluster connection 2020-01-13 22:45:21,341 | test | DEBUG | MainProcess | MainThread | [sdk_client3:_createConn:96] Creating cluster connection 2020-01-13 22:45:21,361 | test | DEBUG | MainProcess | MainThread | [sdk_client3:_createConn:96] Creating cluster connection 2020-01-13 22:45:21,375 | test | DEBUG | MainProcess | MainThread | [sdk_client3:_createConn:96] Creating cluster connection 2020-01-13 22:45:21,391 | test | DEBUG | MainProcess | MainThread | [sdk_client3:_createConn:96] Creating cluster connection 2020-01-13 22:45:21,427 | test | DEBUG | MainProcess | MainThread | [sdk_client3:_createConn:96] Creating cluster connection 2020-01-13 22:45:21,479 | test | INFO | MainProcess | MainThread | [swaprebalancetests:_add_back_failed_node:391] current nodes : [u'ns_1@172.23.120.192', u'ns_1@172.23.120.19', u'ns_1@172.23.120.162', u'ns_1@172.23.120.177'] 2020-01-13 22:45:21,523 | test | INFO | MainProcess | MainThread | [swaprebalancetests:_add_back_failed_node:396] To be removed nodes: [, ] 2020-01-13 22:45:21,523 | test | DEBUG | MainProcess | pool-4-thread-5 | [task:call:2870] Ops rate for 'default': 0.000000 2020-01-13 22:45:21,523 | test | INFO | MainProcess | MainThread | [swaprebalancetests:_add_back_failed_node:409] Node 172.23.120.162:8091 not failed over 2020-01-13 22:45:21,525 | test | INFO | MainProcess | MainThread | [swaprebalancetests:_add_back_failed_node:409] Node 172.23.120.177:8091 not failed over 2020-01-13 22:45:21,525 | test | INFO | MainProcess | MainThread | [swaprebalancetests:_add_back_failed_node:423] To be removed nodes: [u'ns_1@172.23.120.192', u'ns_1@172.23.120.19'] 2020-01-13 22:45:21,525 | test | INFO | MainProcess | MainThread | [swaprebalancetests:_add_back_failed_node:426] Failover node ns_1@172.23.120.192 and rebalance afterwards 2020-01-13 22:45:21,911 | test | DEBUG | MainProcess | pool-4-thread-1 | [task:batch_create:450] Reading values [u'test_docs-00001325'] after failure 2020-01-13 22:45:21,911 | test | DEBUG | MainProcess | pool-4-thread-4 | [task:batch_create:450] Reading values [u'test_docs-00001918', u'test_docs-00001929', u'test_docs-00001930', u'test_docs-00001923'] after failure 2020-01-13 22:45:21,911 | test | DEBUG | MainProcess | pool-4-thread-8 | [task:batch_create:450] Reading values [u'test_docs-00001814', u'test_docs-00001825', u'test_docs-00001811'] after failure 2020-01-13 22:45:21,911 | test | DEBUG | MainProcess | pool-4-thread-2 | [task:batch_create:450] Reading values [u'test_docs-00001579', u'test_docs-00001565', u'test_docs-00001576'] after failure 2020-01-13 22:45:21,913 | test | DEBUG | MainProcess | pool-4-thread-3 | [task:batch_create:450] Reading values [u'test_docs-00001693', u'test_docs-00001699', u'test_docs-00001703'] after failure 2020-01-13 22:45:21,911 | test | DEBUG | MainProcess | pool-4-thread-9 | [task:batch_create:450] Reading values [u'test_docs-00001186', u'test_docs-00001195', u'test_docs-00001189', u'test_docs-00001201', u'test_docs-00001204'] after failure 2020-01-13 22:45:21,914 | test | DEBUG | MainProcess | pool-4-thread-7 | [task:batch_create:450] Reading values [u'test_docs-00001426'] after failure 2020-01-13 22:45:21,914 | test | DEBUG | MainProcess | pool-4-thread-10 | [task:batch_create:450] Reading values [u'test_docs-00001068', u'test_docs-00001067'] after failure 2020-01-13 22:45:22,223 | test | DEBUG | MainProcess | MainThread | [rest_client:fail_over:1262] ns_1@172.23.120.192 - Failover successful 2020-01-13 22:45:22,223 | test | INFO | MainProcess | MainThread | [swaprebalancetests:_add_back_failed_node:426] Failover node ns_1@172.23.120.19 and rebalance afterwards 2020-01-13 22:45:22,480 | test | INFO | MainProcess | pool-4-thread-8 | [table_view:display:72] Failed items: +--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001814 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":35,"idempotent":false,"lastChannelId":"0000000000000023/00000000A90CB053","lastDispatchedFrom":"172.23.97.100:60506","lastDispatchedTo":"172.23.120.192:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":6141,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001814","opaque":"0x2058","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":4,"totalMicros":335362}} | | test_docs-00001825 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":35,"idempotent":false,"lastChannelId":"0000000000000023/00000000A90CB053","lastDispatchedFrom":"172.23.97.100:60506","lastDispatchedTo":"172.23.120.192:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":6142,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001825","opaque":"0x2059","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":4,"totalMicros":335490}} | | test_docs-00001811 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":35,"idempotent":false,"lastChannelId":"0000000000000023/00000000A90CB053","lastDispatchedFrom":"172.23.97.100:60506","lastDispatchedTo":"172.23.120.192:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":6147,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001811","opaque":"0x205e","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":4,"totalMicros":335456}} | +--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:45:22,483 | test | INFO | MainProcess | pool-4-thread-1 | [table_view:display:72] Failed items: +--------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001325 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":31,"idempotent":false,"lastChannelId":"000000000000001F/00000000D43C30B2","lastDispatchedFrom":"172.23.97.100:60438","lastDispatchedTo":"172.23.120.192:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":6053,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001325","opaque":"0x2000","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":11,"totalMicros":344591}} | +--------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:45:22,484 | test | INFO | MainProcess | pool-4-thread-10 | [table_view:display:72] Failed items: +--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001068 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":29,"idempotent":false,"lastChannelId":"000000000000001D/00000000726D67F9","lastDispatchedFrom":"172.23.97.100:60410","lastDispatchedTo":"172.23.120.192:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":6022,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001068","opaque":"0x1fe1","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":4,"totalMicros":347112}} | | test_docs-00001067 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":29,"idempotent":false,"lastChannelId":"000000000000001D/00000000726D67F9","lastDispatchedFrom":"172.23.97.100:60410","lastDispatchedTo":"172.23.120.192:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":6027,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001067","opaque":"0x1fe6","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":5,"totalMicros":347105}} | +--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:45:22,484 | test | INFO | MainProcess | pool-4-thread-2 | [table_view:display:72] Failed items: +--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001579 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":33,"idempotent":false,"lastChannelId":"0000000000000021/00000000B916D0E9","lastDispatchedFrom":"172.23.97.100:60468","lastDispatchedTo":"172.23.120.192:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":6101,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001579","opaque":"0x2030","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":4,"totalMicros":338707}} | | test_docs-00001565 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":33,"idempotent":false,"lastChannelId":"0000000000000021/00000000B916D0E9","lastDispatchedFrom":"172.23.97.100:60468","lastDispatchedTo":"172.23.120.192:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":6106,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001565","opaque":"0x2035","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":4,"totalMicros":338874}} | | test_docs-00001576 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":33,"idempotent":false,"lastChannelId":"0000000000000021/00000000B916D0E9","lastDispatchedFrom":"172.23.97.100:60468","lastDispatchedTo":"172.23.120.192:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":6107,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001576","opaque":"0x2036","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":4,"totalMicros":338973}} | +--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:45:22,486 | test | INFO | MainProcess | pool-4-thread-3 | [table_view:display:72] Failed items: +--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001693 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":34,"idempotent":false,"lastChannelId":"0000000000000022/00000000575EB689","lastDispatchedFrom":"172.23.97.100:60490","lastDispatchedTo":"172.23.120.192:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":6076,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001693","opaque":"0x2017","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":4,"totalMicros":340605}} | | test_docs-00001699 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":34,"idempotent":false,"lastChannelId":"0000000000000022/00000000575EB689","lastDispatchedFrom":"172.23.97.100:60490","lastDispatchedTo":"172.23.120.192:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":6080,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001699","opaque":"0x201b","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":6,"totalMicros":340618}} | | test_docs-00001703 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":34,"idempotent":false,"lastChannelId":"0000000000000022/00000000575EB689","lastDispatchedFrom":"172.23.97.100:60490","lastDispatchedTo":"172.23.120.192:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":6086,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001703","opaque":"0x2021","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":4,"totalMicros":340305}} | +--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:45:22,487 | test | INFO | MainProcess | pool-4-thread-7 | [table_view:display:72] Failed items: +--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001426 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":32,"idempotent":false,"lastChannelId":"0000000000000020/00000000F5750E7A","lastDispatchedFrom":"172.23.97.100:60454","lastDispatchedTo":"172.23.120.192:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":6006,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001426","opaque":"0x1fd1","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":5,"totalMicros":351036}} | +--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:45:22,490 | test | INFO | MainProcess | pool-4-thread-4 | [table_view:display:72] Failed items: +--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001918 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":36,"idempotent":false,"lastChannelId":"0000000000000024/000000007B630B93","lastDispatchedFrom":"172.23.97.100:60520","lastDispatchedTo":"172.23.120.192:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":6037,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001918","opaque":"0x1ff0","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":7,"totalMicros":345574}} | | test_docs-00001929 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":36,"idempotent":false,"lastChannelId":"0000000000000024/000000007B630B93","lastDispatchedFrom":"172.23.97.100:60520","lastDispatchedTo":"172.23.120.192:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":6038,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001929","opaque":"0x1ff1","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":8,"totalMicros":345630}} | | test_docs-00001930 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":36,"idempotent":false,"lastChannelId":"0000000000000024/000000007B630B93","lastDispatchedFrom":"172.23.97.100:60520","lastDispatchedTo":"172.23.120.192:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":6041,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001930","opaque":"0x1ff4","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":8,"totalMicros":345626}} | | test_docs-00001923 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":36,"idempotent":false,"lastChannelId":"0000000000000024/000000007B630B93","lastDispatchedFrom":"172.23.97.100:60520","lastDispatchedTo":"172.23.120.192:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":6044,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001923","opaque":"0x1ff7","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":5,"totalMicros":345635}} | +--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:45:22,490 | test | INFO | MainProcess | pool-4-thread-9 | [table_view:display:72] Failed items: +--------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001186 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":30,"idempotent":false,"lastChannelId":"000000000000001E/00000000BF9869AC","lastDispatchedFrom":"172.23.97.100:60428","lastDispatchedTo":"172.23.120.192:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":6115,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001186","opaque":"0x203e","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":4,"totalMicros":337066}} | | test_docs-00001195 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":30,"idempotent":false,"lastChannelId":"000000000000001E/00000000BF9869AC","lastDispatchedFrom":"172.23.97.100:60428","lastDispatchedTo":"172.23.120.192:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":6118,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001195","opaque":"0x2041","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":10,"totalMicros":336666}} | | test_docs-00001189 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":30,"idempotent":false,"lastChannelId":"000000000000001E/00000000BF9869AC","lastDispatchedFrom":"172.23.97.100:60428","lastDispatchedTo":"172.23.120.192:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":6119,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001189","opaque":"0x2042","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":7,"totalMicros":336718}} | | test_docs-00001201 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":30,"idempotent":false,"lastChannelId":"000000000000001E/00000000BF9869AC","lastDispatchedFrom":"172.23.97.100:60428","lastDispatchedTo":"172.23.120.192:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":6121,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001201","opaque":"0x2044","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":6,"totalMicros":336689}} | | test_docs-00001204 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":30,"idempotent":false,"lastChannelId":"000000000000001E/00000000BF9869AC","lastDispatchedFrom":"172.23.97.100:60428","lastDispatchedTo":"172.23.120.192:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":6126,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001204","opaque":"0x2049","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":6,"totalMicros":336602}} | +--------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:45:22,568 | test | DEBUG | MainProcess | pool-4-thread-5 | [task:call:2870] Ops rate for 'default': 0.000000 2020-01-13 22:45:22,615 | test | DEBUG | MainProcess | pool-4-thread-8 | [task:batch_create:450] Reading values [u'test_docs-00001839', u'test_docs-00001848', u'test_docs-00001830', u'test_docs-00001835', u'test_docs-00001833'] after failure 2020-01-13 22:45:22,615 | test | DEBUG | MainProcess | pool-4-thread-3 | [task:batch_create:450] Reading values [u'test_docs-00001719', u'test_docs-00001708', u'test_docs-00001718', u'test_docs-00001706', u'test_docs-00001705', u'test_docs-00001709', u'test_docs-00001710', u'test_docs-00001715', u'test_docs-00001713'] after failure 2020-01-13 22:45:22,615 | test | DEBUG | MainProcess | pool-4-thread-4 | [task:batch_create:450] Reading values [u'test_docs-00001939', u'test_docs-00001937', u'test_docs-00001936', u'test_docs-00001947', u'test_docs-00001942', u'test_docs-00001941', u'test_docs-00001952', u'test_docs-00001951', u'test_docs-00001935', u'test_docs-00001944', u'test_docs-00001943', u'test_docs-00001954'] after failure 2020-01-13 22:45:22,618 | test | INFO | MainProcess | pool-4-thread-8 | [table_view:display:72] Failed items: +--------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001839 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":35,"idempotent":false,"lastChannelId":"0000000000000023/000000006B0D5797","lastDispatchedFrom":"172.23.97.100:42886","lastDispatchedTo":"172.23.120.177:11210","requestId":6219,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001839","opaque":"0x2194","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":4172,"encodingMicros":16,"totalMicros":4332,"serverMicros":0}} | | test_docs-00001848 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":35,"idempotent":false,"lastChannelId":"0000000000000023/00000000A5F5D7E0","lastDispatchedFrom":"172.23.97.100:38430","lastDispatchedTo":"172.23.120.162:11210","requestId":6223,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001848","opaque":"0x2198","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":9021,"encodingMicros":7,"totalMicros":9311,"serverMicros":0}} | | test_docs-00001830 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":35,"idempotent":false,"lastChannelId":"0000000000000023/00000000A5F5D7E0","lastDispatchedFrom":"172.23.97.100:38430","lastDispatchedTo":"172.23.120.162:11210","requestId":6228,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001830","opaque":"0x219d","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":13206,"encodingMicros":4,"totalMicros":13402,"serverMicros":0}} | | test_docs-00001835 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":35,"idempotent":false,"lastChannelId":"0000000000000023/00000000A5F5D7E0","lastDispatchedFrom":"172.23.97.100:38430","lastDispatchedTo":"172.23.120.162:11210","requestId":6233,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001835","opaque":"0x21a2","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":121661,"encodingMicros":4,"totalMicros":121785,"serverMicros":0}} | | test_docs-00001833 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":35,"idempotent":false,"lastChannelId":"0000000000000023/00000000A5F5D7E0","lastDispatchedFrom":"172.23.97.100:38430","lastDispatchedTo":"172.23.120.162:11210","requestId":6237,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001833","opaque":"0x21a6","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":124337,"encodingMicros":4,"totalMicros":124449,"serverMicros":0}} | +--------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:45:22,618 | test | INFO | MainProcess | pool-4-thread-3 | [table_view:display:72] Failed items: +--------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001719 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":34,"idempotent":false,"lastChannelId":"0000000000000022/00000000F39B8058","lastDispatchedFrom":"172.23.97.100:42870","lastDispatchedTo":"172.23.120.177:11210","requestId":6300,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001719","opaque":"0x21e6","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":1064,"encodingMicros":6,"totalMicros":1259,"serverMicros":0}} | | test_docs-00001708 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":34,"idempotent":false,"lastChannelId":"0000000000000022/000000009BA0D650","lastDispatchedFrom":"172.23.97.100:38408","lastDispatchedTo":"172.23.120.162:11210","requestId":6299,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001708","opaque":"0x21e5","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":2766,"encodingMicros":12,"totalMicros":2991,"serverMicros":0}} | | test_docs-00001718 | com.couchbase.client.core.error.DurabilityAmbiguousException: The server returned with a durability ambiguous response on this request {"completed":true,"coreId":34,"idempotent":false,"lastChannelId":"0000000000000022/00000000F39B8058","lastDispatchedFrom":"172.23.97.100:42870","lastDispatchedTo":"172.23.120.177:11210","requestId":6302,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001718","opaque":"0x21e8","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"SYNC_WRITE_AMBIGUOUS","timeoutMs":5000,"timings":{"dispatchMicros":1272,"encodingMicros":4,"totalMicros":1423,"serverMicros":0}} | | test_docs-00001706 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":34,"idempotent":false,"lastChannelId":"0000000000000022/000000009BA0D650","lastDispatchedFrom":"172.23.97.100:38408","lastDispatchedTo":"172.23.120.162:11210","requestId":6303,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001706","opaque":"0x21e9","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":3049,"encodingMicros":4,"totalMicros":3166,"serverMicros":0}} | | test_docs-00001705 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":34,"idempotent":false,"lastChannelId":"0000000000000022/000000009BA0D650","lastDispatchedFrom":"172.23.97.100:38408","lastDispatchedTo":"172.23.120.162:11210","requestId":6305,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001705","opaque":"0x21eb","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":3086,"encodingMicros":8,"totalMicros":3219,"serverMicros":0}} | | test_docs-00001709 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":34,"idempotent":false,"lastChannelId":"0000000000000022/00000000F39B8058","lastDispatchedFrom":"172.23.97.100:42870","lastDispatchedTo":"172.23.120.177:11210","requestId":6307,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001709","opaque":"0x21ed","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":99346,"encodingMicros":8,"totalMicros":99708,"serverMicros":0}} | | test_docs-00001710 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":34,"idempotent":false,"lastChannelId":"0000000000000022/000000009BA0D650","lastDispatchedFrom":"172.23.97.100:38408","lastDispatchedTo":"172.23.120.162:11210","requestId":6310,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001710","opaque":"0x21f1","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":114882,"encodingMicros":4,"totalMicros":114977,"serverMicros":0}} | | test_docs-00001715 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":34,"idempotent":false,"lastChannelId":"0000000000000022/000000009BA0D650","lastDispatchedFrom":"172.23.97.100:38408","lastDispatchedTo":"172.23.120.162:11210","requestId":6313,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001715","opaque":"0x21f4","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":116014,"encodingMicros":24,"totalMicros":118632,"serverMicros":0}} | | test_docs-00001713 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":34,"idempotent":false,"lastChannelId":"0000000000000022/000000009BA0D650","lastDispatchedFrom":"172.23.97.100:38408","lastDispatchedTo":"172.23.120.162:11210","requestId":6315,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001713","opaque":"0x21f6","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":118276,"encodingMicros":8,"totalMicros":120858,"serverMicros":0}} | +--------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:45:22,618 | test | INFO | MainProcess | pool-4-thread-4 | [table_view:display:72] Failed items: +--------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001939 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":36,"idempotent":false,"lastChannelId":"0000000000000024/0000000094F818B4","lastDispatchedFrom":"172.23.97.100:38446","lastDispatchedTo":"172.23.120.162:11210","requestId":6339,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001939","opaque":"0x220e","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":505,"encodingMicros":13,"totalMicros":693,"serverMicros":0}} | | test_docs-00001937 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":36,"idempotent":false,"lastChannelId":"0000000000000024/00000000C53D09DD","lastDispatchedFrom":"172.23.97.100:42898","lastDispatchedTo":"172.23.120.177:11210","requestId":6342,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001937","opaque":"0x2211","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":3737,"encodingMicros":5,"totalMicros":4109,"serverMicros":0}} | | test_docs-00001936 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":36,"idempotent":false,"lastChannelId":"0000000000000024/00000000C53D09DD","lastDispatchedFrom":"172.23.97.100:42898","lastDispatchedTo":"172.23.120.177:11210","requestId":6344,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001936","opaque":"0x2213","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":3834,"encodingMicros":5,"totalMicros":4153,"serverMicros":0}} | | test_docs-00001947 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":36,"idempotent":false,"lastChannelId":"0000000000000024/0000000094F818B4","lastDispatchedFrom":"172.23.97.100:38446","lastDispatchedTo":"172.23.120.162:11210","requestId":6345,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001947","opaque":"0x2214","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":109769,"encodingMicros":4,"totalMicros":109920,"serverMicros":0}} | | test_docs-00001942 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":36,"idempotent":false,"lastChannelId":"0000000000000024/00000000C53D09DD","lastDispatchedFrom":"172.23.97.100:42898","lastDispatchedTo":"172.23.120.177:11210","requestId":6346,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001942","opaque":"0x2215","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":3885,"encodingMicros":4,"totalMicros":4170,"serverMicros":0}} | | test_docs-00001941 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":36,"idempotent":false,"lastChannelId":"0000000000000024/0000000094F818B4","lastDispatchedFrom":"172.23.97.100:38446","lastDispatchedTo":"172.23.120.162:11210","requestId":6348,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001941","opaque":"0x2217","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":112030,"encodingMicros":4,"totalMicros":112116,"serverMicros":0}} | | test_docs-00001952 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":36,"idempotent":false,"lastChannelId":"0000000000000024/0000000094F818B4","lastDispatchedFrom":"172.23.97.100:38446","lastDispatchedTo":"172.23.120.162:11210","requestId":6349,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001952","opaque":"0x2218","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":112099,"encodingMicros":6,"totalMicros":112159,"serverMicros":0}} | | test_docs-00001951 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":36,"idempotent":false,"lastChannelId":"0000000000000024/0000000094F818B4","lastDispatchedFrom":"172.23.97.100:38446","lastDispatchedTo":"172.23.120.162:11210","requestId":6352,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001951","opaque":"0x221b","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":113457,"encodingMicros":5,"totalMicros":113529,"serverMicros":0}} | | test_docs-00001935 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":36,"idempotent":false,"lastChannelId":"0000000000000024/00000000C53D09DD","lastDispatchedFrom":"172.23.97.100:42898","lastDispatchedTo":"172.23.120.177:11210","requestId":6356,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001935","opaque":"0x221f","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":5032,"encodingMicros":6,"totalMicros":5201,"serverMicros":0}} | | test_docs-00001944 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":36,"idempotent":false,"lastChannelId":"0000000000000024/0000000094F818B4","lastDispatchedFrom":"172.23.97.100:38446","lastDispatchedTo":"172.23.120.162:11210","requestId":6361,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001944","opaque":"0x2224","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":115677,"encodingMicros":7,"totalMicros":115786,"serverMicros":0}} | | test_docs-00001943 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":36,"idempotent":false,"lastChannelId":"0000000000000024/00000000C53D09DD","lastDispatchedFrom":"172.23.97.100:42898","lastDispatchedTo":"172.23.120.177:11210","requestId":6364,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001943","opaque":"0x2227","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":6013,"encodingMicros":25,"totalMicros":6096,"serverMicros":0}} | | test_docs-00001954 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":36,"idempotent":false,"lastChannelId":"0000000000000024/00000000C53D09DD","lastDispatchedFrom":"172.23.97.100:42898","lastDispatchedTo":"172.23.120.177:11210","requestId":6366,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001954","opaque":"0x2229","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":6042,"encodingMicros":12,"totalMicros":6122,"serverMicros":0}} | +--------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:45:22,621 | test | DEBUG | MainProcess | pool-4-thread-2 | [task:batch_create:450] Reading values [u'test_docs-00001589', u'test_docs-00001588', u'test_docs-00001598'] after failure 2020-01-13 22:45:22,625 | test | INFO | MainProcess | pool-4-thread-2 | [table_view:display:72] Failed items: +--------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001589 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":33,"idempotent":false,"lastChannelId":"0000000000000021/00000000A73830BD","lastDispatchedFrom":"172.23.97.100:38398","lastDispatchedTo":"172.23.120.162:11210","requestId":6294,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001589","opaque":"0x21df","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":125540,"encodingMicros":4,"totalMicros":125726,"serverMicros":0}} | | test_docs-00001588 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":33,"idempotent":false,"lastChannelId":"0000000000000021/0000000085C4D4FC","lastDispatchedFrom":"172.23.97.100:42846","lastDispatchedTo":"172.23.120.177:11210","requestId":6295,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001588","opaque":"0x21e0","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":114127,"encodingMicros":4,"totalMicros":114598,"serverMicros":0}} | | test_docs-00001598 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":33,"idempotent":false,"lastChannelId":"0000000000000021/00000000A73830BD","lastDispatchedFrom":"172.23.97.100:38398","lastDispatchedTo":"172.23.120.162:11210","requestId":6298,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001598","opaque":"0x21e4","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":126567,"encodingMicros":4,"totalMicros":126688,"serverMicros":0}} | +--------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:45:22,644 | test | DEBUG | MainProcess | pool-4-thread-2 | [task:batch_create:450] Reading values [u'test_docs-00001609', u'test_docs-00001607', u'test_docs-00001606', u'test_docs-00001617', u'test_docs-00001601', u'test_docs-00001612', u'test_docs-00001600', u'test_docs-00001611', u'test_docs-00001610', u'test_docs-00001605', u'test_docs-00001616', u'test_docs-00001615', u'test_docs-00001604', u'test_docs-00001603', u'test_docs-00001614', u'test_docs-00001602', u'test_docs-00001613'] after failure 2020-01-13 22:45:22,644 | test | DEBUG | MainProcess | pool-4-thread-4 | [task:batch_create:450] Reading values [u'test_docs-00001969', u'test_docs-00001960', u'test_docs-00001971', u'test_docs-00001970', u'test_docs-00001964', u'test_docs-00001963', u'test_docs-00001974', u'test_docs-00001973', u'test_docs-00001962', u'test_docs-00001961', u'test_docs-00001972', u'test_docs-00001957', u'test_docs-00001967', u'test_docs-00001955', u'test_docs-00001966', u'test_docs-00001965'] after failure 2020-01-13 22:45:22,647 | test | INFO | MainProcess | pool-4-thread-2 | [table_view:display:72] Failed items: +--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001609 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":33,"idempotent":false,"lastChannelId":"0000000000000021/00000000A73830BD","lastDispatchedFrom":"172.23.97.100:38398","lastDispatchedTo":"172.23.120.162:11210","requestId":6468,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001609","opaque":"0x228f","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":1376,"encodingMicros":13,"totalMicros":1525,"serverMicros":0}} | | test_docs-00001607 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":33,"idempotent":false,"lastChannelId":"0000000000000021/0000000085C4D4FC","lastDispatchedFrom":"172.23.97.100:42846","lastDispatchedTo":"172.23.120.177:11210","requestId":6471,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001607","opaque":"0x2292","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":6737,"encodingMicros":20,"totalMicros":6835,"serverMicros":0}} | | test_docs-00001606 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":33,"idempotent":false,"lastChannelId":"0000000000000021/0000000085C4D4FC","lastDispatchedFrom":"172.23.97.100:42846","lastDispatchedTo":"172.23.120.177:11210","requestId":6473,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001606","opaque":"0x2294","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":6748,"encodingMicros":5,"totalMicros":6833,"serverMicros":0}} | | test_docs-00001617 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":33,"idempotent":false,"lastChannelId":"0000000000000021/00000000A73830BD","lastDispatchedFrom":"172.23.97.100:38398","lastDispatchedTo":"172.23.120.162:11210","requestId":6474,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001617","opaque":"0x2295","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":7842,"encodingMicros":4,"totalMicros":9264,"serverMicros":0}} | | test_docs-00001601 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":33,"idempotent":false,"lastChannelId":"0000000000000021/00000000A73830BD","lastDispatchedFrom":"172.23.97.100:38398","lastDispatchedTo":"172.23.120.162:11210","requestId":6475,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001601","opaque":"0x2296","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":7904,"encodingMicros":4,"totalMicros":9292,"serverMicros":0}} | | test_docs-00001612 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":33,"idempotent":false,"lastChannelId":"0000000000000021/00000000A73830BD","lastDispatchedFrom":"172.23.97.100:38398","lastDispatchedTo":"172.23.120.162:11210","requestId":6476,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001612","opaque":"0x2297","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":7939,"encodingMicros":4,"totalMicros":9316,"serverMicros":0}} | | test_docs-00001600 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":33,"idempotent":false,"lastChannelId":"0000000000000021/0000000085C4D4FC","lastDispatchedFrom":"172.23.97.100:42846","lastDispatchedTo":"172.23.120.177:11210","requestId":6477,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001600","opaque":"0x2298","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":6804,"encodingMicros":4,"totalMicros":6869,"serverMicros":0}} | | test_docs-00001611 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":33,"idempotent":false,"lastChannelId":"0000000000000021/0000000085C4D4FC","lastDispatchedFrom":"172.23.97.100:42846","lastDispatchedTo":"172.23.120.177:11210","requestId":6478,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001611","opaque":"0x2299","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":6851,"encodingMicros":4,"totalMicros":6918,"serverMicros":0}} | | test_docs-00001610 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":33,"idempotent":false,"lastChannelId":"0000000000000021/0000000085C4D4FC","lastDispatchedFrom":"172.23.97.100:42846","lastDispatchedTo":"172.23.120.177:11210","requestId":6479,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001610","opaque":"0x229a","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":6886,"encodingMicros":4,"totalMicros":6958,"serverMicros":0}} | | test_docs-00001605 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":33,"idempotent":false,"lastChannelId":"0000000000000021/0000000085C4D4FC","lastDispatchedFrom":"172.23.97.100:42846","lastDispatchedTo":"172.23.120.177:11210","requestId":6480,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001605","opaque":"0x229b","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":6930,"encodingMicros":4,"totalMicros":6975,"serverMicros":0}} | | test_docs-00001616 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":33,"idempotent":false,"lastChannelId":"0000000000000021/0000000085C4D4FC","lastDispatchedFrom":"172.23.97.100:42846","lastDispatchedTo":"172.23.120.177:11210","requestId":6481,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001616","opaque":"0x229c","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":6957,"encodingMicros":4,"totalMicros":6996,"serverMicros":0}} | | test_docs-00001615 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":33,"idempotent":false,"lastChannelId":"0000000000000021/0000000085C4D4FC","lastDispatchedFrom":"172.23.97.100:42846","lastDispatchedTo":"172.23.120.177:11210","requestId":6483,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001615","opaque":"0x229e","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":6941,"encodingMicros":4,"totalMicros":7011,"serverMicros":0}} | | test_docs-00001604 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":33,"idempotent":false,"lastChannelId":"0000000000000021/00000000A73830BD","lastDispatchedFrom":"172.23.97.100:38398","lastDispatchedTo":"172.23.120.162:11210","requestId":6482,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001604","opaque":"0x229d","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":7987,"encodingMicros":4,"totalMicros":9268,"serverMicros":0}} | | test_docs-00001603 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":33,"idempotent":false,"lastChannelId":"0000000000000021/0000000085C4D4FC","lastDispatchedFrom":"172.23.97.100:42846","lastDispatchedTo":"172.23.120.177:11210","requestId":6484,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001603","opaque":"0x229f","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":6953,"encodingMicros":8,"totalMicros":7028,"serverMicros":0}} | | test_docs-00001614 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":33,"idempotent":false,"lastChannelId":"0000000000000021/0000000085C4D4FC","lastDispatchedFrom":"172.23.97.100:42846","lastDispatchedTo":"172.23.120.177:11210","requestId":6485,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001614","opaque":"0x22a0","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":6961,"encodingMicros":6,"totalMicros":7026,"serverMicros":0}} | | test_docs-00001602 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":33,"idempotent":false,"lastChannelId":"0000000000000021/0000000085C4D4FC","lastDispatchedFrom":"172.23.97.100:42846","lastDispatchedTo":"172.23.120.177:11210","requestId":6486,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001602","opaque":"0x22a1","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":6993,"encodingMicros":4,"totalMicros":7035,"serverMicros":0}} | | test_docs-00001613 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":33,"idempotent":false,"lastChannelId":"0000000000000021/0000000085C4D4FC","lastDispatchedFrom":"172.23.97.100:42846","lastDispatchedTo":"172.23.120.177:11210","requestId":6487,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001613","opaque":"0x22a2","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":6969,"encodingMicros":4,"totalMicros":7056,"serverMicros":0}} | +--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:45:22,648 | test | INFO | MainProcess | pool-4-thread-4 | [table_view:display:72] Failed items: +--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001969 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":36,"idempotent":false,"lastChannelId":"0000000000000024/0000000094F818B4","lastDispatchedFrom":"172.23.97.100:38446","lastDispatchedTo":"172.23.120.162:11210","requestId":6450,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001969","opaque":"0x227d","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":3621,"encodingMicros":6,"totalMicros":3921,"serverMicros":0}} | | test_docs-00001960 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":36,"idempotent":false,"lastChannelId":"0000000000000024/00000000C53D09DD","lastDispatchedFrom":"172.23.97.100:42898","lastDispatchedTo":"172.23.120.177:11210","requestId":6451,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001960","opaque":"0x227e","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":268,"encodingMicros":5,"totalMicros":408,"serverMicros":0}} | | test_docs-00001971 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":36,"idempotent":false,"lastChannelId":"0000000000000024/0000000094F818B4","lastDispatchedFrom":"172.23.97.100:38446","lastDispatchedTo":"172.23.120.162:11210","requestId":6452,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001971","opaque":"0x227f","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":3706,"encodingMicros":6,"totalMicros":3910,"serverMicros":0}} | | test_docs-00001970 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":36,"idempotent":false,"lastChannelId":"0000000000000024/00000000C53D09DD","lastDispatchedFrom":"172.23.97.100:42898","lastDispatchedTo":"172.23.120.177:11210","requestId":6453,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001970","opaque":"0x2280","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":423,"encodingMicros":5,"totalMicros":498,"serverMicros":0}} | | test_docs-00001964 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":36,"idempotent":false,"lastChannelId":"0000000000000024/00000000C53D09DD","lastDispatchedFrom":"172.23.97.100:42898","lastDispatchedTo":"172.23.120.177:11210","requestId":6454,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001964","opaque":"0x2281","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":454,"encodingMicros":6,"totalMicros":519,"serverMicros":0}} | | test_docs-00001963 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":36,"idempotent":false,"lastChannelId":"0000000000000024/00000000C53D09DD","lastDispatchedFrom":"172.23.97.100:42898","lastDispatchedTo":"172.23.120.177:11210","requestId":6455,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001963","opaque":"0x2282","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":489,"encodingMicros":6,"totalMicros":563,"serverMicros":0}} | | test_docs-00001974 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":36,"idempotent":false,"lastChannelId":"0000000000000024/0000000094F818B4","lastDispatchedFrom":"172.23.97.100:38446","lastDispatchedTo":"172.23.120.162:11210","requestId":6456,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001974","opaque":"0x2283","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":3744,"encodingMicros":5,"totalMicros":3851,"serverMicros":0}} | | test_docs-00001973 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":36,"idempotent":false,"lastChannelId":"0000000000000024/00000000C53D09DD","lastDispatchedFrom":"172.23.97.100:42898","lastDispatchedTo":"172.23.120.177:11210","requestId":6458,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001973","opaque":"0x2285","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":429,"encodingMicros":6,"totalMicros":543,"serverMicros":0}} | | test_docs-00001962 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":36,"idempotent":false,"lastChannelId":"0000000000000024/0000000094F818B4","lastDispatchedFrom":"172.23.97.100:38446","lastDispatchedTo":"172.23.120.162:11210","requestId":6457,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001962","opaque":"0x2284","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":3777,"encodingMicros":6,"totalMicros":3867,"serverMicros":0}} | | test_docs-00001961 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":36,"idempotent":false,"lastChannelId":"0000000000000024/00000000C53D09DD","lastDispatchedFrom":"172.23.97.100:42898","lastDispatchedTo":"172.23.120.177:11210","requestId":6459,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001961","opaque":"0x2286","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":465,"encodingMicros":6,"totalMicros":549,"serverMicros":0}} | | test_docs-00001972 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":36,"idempotent":false,"lastChannelId":"0000000000000024/00000000C53D09DD","lastDispatchedFrom":"172.23.97.100:42898","lastDispatchedTo":"172.23.120.177:11210","requestId":6460,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001972","opaque":"0x2287","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":525,"encodingMicros":6,"totalMicros":600,"serverMicros":0}} | | test_docs-00001957 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":36,"idempotent":false,"lastChannelId":"0000000000000024/00000000C53D09DD","lastDispatchedFrom":"172.23.97.100:42898","lastDispatchedTo":"172.23.120.177:11210","requestId":6461,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001957","opaque":"0x2288","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":552,"encodingMicros":6,"totalMicros":617,"serverMicros":0}} | | test_docs-00001967 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":36,"idempotent":false,"lastChannelId":"0000000000000024/0000000094F818B4","lastDispatchedFrom":"172.23.97.100:38446","lastDispatchedTo":"172.23.120.162:11210","requestId":6464,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001967","opaque":"0x228b","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":12846,"encodingMicros":6,"totalMicros":13587,"serverMicros":0}} | | test_docs-00001955 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":36,"idempotent":false,"lastChannelId":"0000000000000024/00000000C53D09DD","lastDispatchedFrom":"172.23.97.100:42898","lastDispatchedTo":"172.23.120.177:11210","requestId":6465,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001955","opaque":"0x228c","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":443,"encodingMicros":33,"totalMicros":520,"serverMicros":0}} | | test_docs-00001966 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":36,"idempotent":false,"lastChannelId":"0000000000000024/00000000C53D09DD","lastDispatchedFrom":"172.23.97.100:42898","lastDispatchedTo":"172.23.120.177:11210","requestId":6466,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001966","opaque":"0x228d","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":479,"encodingMicros":6,"totalMicros":531,"serverMicros":0}} | | test_docs-00001965 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":36,"idempotent":false,"lastChannelId":"0000000000000024/00000000C53D09DD","lastDispatchedFrom":"172.23.97.100:42898","lastDispatchedTo":"172.23.120.177:11210","requestId":6467,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001965","opaque":"0x228e","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":482,"encodingMicros":6,"totalMicros":546,"serverMicros":0}} | +--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:45:22,654 | test | DEBUG | MainProcess | pool-4-thread-3 | [task:batch_create:450] Reading values [u'test_docs-00001728', u'test_docs-00001738', u'test_docs-00001740', u'test_docs-00001743', u'test_docs-00001725'] after failure 2020-01-13 22:45:22,654 | test | DEBUG | MainProcess | pool-4-thread-2 | [task:batch_create:450] Reading values [u'test_docs-00001623', u'test_docs-00001622', u'test_docs-00001621', u'test_docs-00001620', u'test_docs-00001624'] after failure 2020-01-13 22:45:22,657 | test | DEBUG | MainProcess | pool-4-thread-8 | [task:batch_create:450] Reading values [u'test_docs-00001859', u'test_docs-00001865', u'test_docs-00001863', u'test_docs-00001869', u'test_docs-00001858', u'test_docs-00001868', u'test_docs-00001866'] after failure 2020-01-13 22:45:22,657 | test | INFO | MainProcess | pool-4-thread-3 | [table_view:display:72] Failed items: +--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001728 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":34,"idempotent":false,"lastChannelId":"0000000000000022/000000009BA0D650","lastDispatchedFrom":"172.23.97.100:38408","lastDispatchedTo":"172.23.120.162:11210","requestId":6426,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001728","opaque":"0x2265","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":587,"encodingMicros":6,"totalMicros":713,"serverMicros":0}} | | test_docs-00001738 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":34,"idempotent":false,"lastChannelId":"0000000000000022/000000009BA0D650","lastDispatchedFrom":"172.23.97.100:38408","lastDispatchedTo":"172.23.120.162:11210","requestId":6429,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001738","opaque":"0x2268","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":3760,"encodingMicros":4,"totalMicros":3845,"serverMicros":0}} | | test_docs-00001740 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":34,"idempotent":false,"lastChannelId":"0000000000000022/000000009BA0D650","lastDispatchedFrom":"172.23.97.100:38408","lastDispatchedTo":"172.23.120.162:11210","requestId":6430,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001740","opaque":"0x2269","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":3842,"encodingMicros":4,"totalMicros":3896,"serverMicros":0}} | | test_docs-00001743 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":34,"idempotent":false,"lastChannelId":"0000000000000022/000000009BA0D650","lastDispatchedFrom":"172.23.97.100:38408","lastDispatchedTo":"172.23.120.162:11210","requestId":6434,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001743","opaque":"0x226d","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":13445,"encodingMicros":4,"totalMicros":13523,"serverMicros":0}} | | test_docs-00001725 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":34,"idempotent":false,"lastChannelId":"0000000000000022/000000009BA0D650","lastDispatchedFrom":"172.23.97.100:38408","lastDispatchedTo":"172.23.120.162:11210","requestId":6441,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001725","opaque":"0x2274","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":22151,"encodingMicros":4,"totalMicros":22242,"serverMicros":0}} | +--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:45:22,657 | test | INFO | MainProcess | pool-4-thread-2 | [table_view:display:72] Failed items: +--------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001623 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":33,"idempotent":false,"lastChannelId":"0000000000000021/00000000A73830BD","lastDispatchedFrom":"172.23.97.100:38398","lastDispatchedTo":"172.23.120.162:11210","requestId":6521,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001623","opaque":"0x22c4","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":946,"encodingMicros":12,"totalMicros":1102,"serverMicros":0}} | | test_docs-00001622 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":33,"idempotent":false,"lastChannelId":"0000000000000021/00000000A73830BD","lastDispatchedFrom":"172.23.97.100:38398","lastDispatchedTo":"172.23.120.162:11210","requestId":6522,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001622","opaque":"0x22c5","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":1026,"encodingMicros":7,"totalMicros":1100,"serverMicros":0}} | | test_docs-00001621 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":33,"idempotent":false,"lastChannelId":"0000000000000021/00000000A73830BD","lastDispatchedFrom":"172.23.97.100:38398","lastDispatchedTo":"172.23.120.162:11210","requestId":6523,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001621","opaque":"0x22c6","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":1041,"encodingMicros":5,"totalMicros":1117,"serverMicros":0}} | | test_docs-00001620 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":33,"idempotent":false,"lastChannelId":"0000000000000021/0000000085C4D4FC","lastDispatchedFrom":"172.23.97.100:42846","lastDispatchedTo":"172.23.120.177:11210","requestId":6524,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001620","opaque":"0x22c7","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":309,"encodingMicros":4,"totalMicros":427,"serverMicros":0}} | | test_docs-00001624 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":33,"idempotent":false,"lastChannelId":"0000000000000021/00000000A73830BD","lastDispatchedFrom":"172.23.97.100:38398","lastDispatchedTo":"172.23.120.162:11210","requestId":6525,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001624","opaque":"0x22c8","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":1088,"encodingMicros":4,"totalMicros":1139,"serverMicros":0}} | +--------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:45:22,660 | test | INFO | MainProcess | pool-4-thread-8 | [table_view:display:72] Failed items: +--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001859 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":35,"idempotent":false,"lastChannelId":"0000000000000023/000000006B0D5797","lastDispatchedFrom":"172.23.97.100:42886","lastDispatchedTo":"172.23.120.177:11210","requestId":6405,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001859","opaque":"0x2250","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":296,"encodingMicros":19,"totalMicros":470,"serverMicros":0}} | | test_docs-00001865 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":35,"idempotent":false,"lastChannelId":"0000000000000023/00000000A5F5D7E0","lastDispatchedFrom":"172.23.97.100:38430","lastDispatchedTo":"172.23.120.162:11210","requestId":6410,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001865","opaque":"0x2255","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":13897,"encodingMicros":5,"totalMicros":13995,"serverMicros":0}} | | test_docs-00001863 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":35,"idempotent":false,"lastChannelId":"0000000000000023/00000000A5F5D7E0","lastDispatchedFrom":"172.23.97.100:38430","lastDispatchedTo":"172.23.120.162:11210","requestId":6414,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001863","opaque":"0x2259","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":22698,"encodingMicros":4,"totalMicros":22784,"serverMicros":0}} | | test_docs-00001869 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":35,"idempotent":false,"lastChannelId":"0000000000000023/000000006B0D5797","lastDispatchedFrom":"172.23.97.100:42886","lastDispatchedTo":"172.23.120.177:11210","requestId":6418,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001869","opaque":"0x225d","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":13988,"encodingMicros":5,"totalMicros":14049,"serverMicros":0}} | | test_docs-00001858 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":35,"idempotent":false,"lastChannelId":"0000000000000023/00000000A5F5D7E0","lastDispatchedFrom":"172.23.97.100:38430","lastDispatchedTo":"172.23.120.162:11210","requestId":6417,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001858","opaque":"0x225c","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":28164,"encodingMicros":4,"totalMicros":28252,"serverMicros":0}} | | test_docs-00001868 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":35,"idempotent":false,"lastChannelId":"0000000000000023/000000006B0D5797","lastDispatchedFrom":"172.23.97.100:42886","lastDispatchedTo":"172.23.120.177:11210","requestId":6420,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001868","opaque":"0x225f","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":16226,"encodingMicros":5,"totalMicros":16411,"serverMicros":0}} | | test_docs-00001866 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":35,"idempotent":false,"lastChannelId":"0000000000000023/00000000A5F5D7E0","lastDispatchedFrom":"172.23.97.100:38430","lastDispatchedTo":"172.23.120.162:11210","requestId":6424,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001866","opaque":"0x2263","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":29819,"encodingMicros":8,"totalMicros":29895,"serverMicros":0}} | +--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:45:22,667 | test | DEBUG | MainProcess | MainThread | [rest_client:fail_over:1262] ns_1@172.23.120.19 - Failover successful 2020-01-13 22:45:22,673 | test | DEBUG | MainProcess | pool-4-thread-8 | [task:batch_create:450] Reading values [u'test_docs-00001870', u'test_docs-00001873'] after failure 2020-01-13 22:45:22,674 | test | INFO | MainProcess | pool-4-thread-8 | [table_view:display:72] Failed items: +--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001870 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":35,"idempotent":false,"lastChannelId":"0000000000000023/00000000A5F5D7E0","lastDispatchedFrom":"172.23.97.100:38430","lastDispatchedTo":"172.23.120.162:11210","requestId":6570,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001870","opaque":"0x22f5","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":3420,"encodingMicros":5,"totalMicros":3494,"serverMicros":0}} | | test_docs-00001873 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":35,"idempotent":false,"lastChannelId":"0000000000000023/00000000A5F5D7E0","lastDispatchedFrom":"172.23.97.100:38430","lastDispatchedTo":"172.23.120.162:11210","requestId":6572,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001873","opaque":"0x22f7","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":3396,"encodingMicros":27,"totalMicros":3490,"serverMicros":0}} | +--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:45:22,697 | test | DEBUG | MainProcess | pool-4-thread-3 | [task:batch_create:450] Reading values [u'test_docs-00001749', u'test_docs-00001748'] after failure 2020-01-13 22:45:22,700 | test | INFO | MainProcess | pool-4-thread-3 | [table_view:display:72] Failed items: +--------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001749 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":34,"idempotent":false,"lastChannelId":"0000000000000022/00000000F39B8058","lastDispatchedFrom":"172.23.97.100:42870","lastDispatchedTo":"172.23.120.177:11210","requestId":6563,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001749","opaque":"0x22ee","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":244,"encodingMicros":17,"totalMicros":369,"serverMicros":0}} | | test_docs-00001748 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":34,"idempotent":false,"lastChannelId":"0000000000000022/000000009BA0D650","lastDispatchedFrom":"172.23.97.100:38408","lastDispatchedTo":"172.23.120.162:11210","requestId":6564,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001748","opaque":"0x22ef","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":388,"encodingMicros":6,"totalMicros":500,"serverMicros":0}} | +--------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:45:22,703 | test | DEBUG | MainProcess | MainThread | [rest_client:rebalance:1327] Rebalance params: {'password': 'password', 'knownNodes': u'ns_1@172.23.120.192,ns_1@172.23.120.19,ns_1@172.23.120.162,ns_1@172.23.120.177', 'ejectedNodes': u'ns_1@172.23.120.192,ns_1@172.23.120.19', 'user': 'Administrator'} 2020-01-13 22:45:22,713 | test | DEBUG | MainProcess | pool-4-thread-4 | [task:batch_create:450] Reading values [u'test_docs-00001975', u'test_docs-00001988', u'test_docs-00001977', u'test_docs-00001976'] after failure 2020-01-13 22:45:22,713 | test | DEBUG | MainProcess | MainThread | [rest_client:rebalance:1332] Rebalance operation started 2020-01-13 22:45:22,714 | test | INFO | MainProcess | pool-4-thread-4 | [table_view:display:72] Failed items: +--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001975 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":36,"idempotent":false,"lastChannelId":"0000000000000024/00000000C53D09DD","lastDispatchedFrom":"172.23.97.100:42898","lastDispatchedTo":"172.23.120.177:11210","requestId":6533,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001975","opaque":"0x22d0","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":20503,"encodingMicros":5,"totalMicros":20596,"serverMicros":0}} | | test_docs-00001988 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":36,"idempotent":false,"lastChannelId":"0000000000000024/0000000094F818B4","lastDispatchedFrom":"172.23.97.100:38446","lastDispatchedTo":"172.23.120.162:11210","requestId":6543,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001988","opaque":"0x22da","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":27294,"encodingMicros":12,"totalMicros":27410,"serverMicros":0}} | | test_docs-00001977 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":36,"idempotent":false,"lastChannelId":"0000000000000024/00000000C53D09DD","lastDispatchedFrom":"172.23.97.100:42898","lastDispatchedTo":"172.23.120.177:11210","requestId":6542,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001977","opaque":"0x22d9","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":53831,"encodingMicros":8,"totalMicros":53954,"serverMicros":0}} | | test_docs-00001976 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":36,"idempotent":false,"lastChannelId":"0000000000000024/00000000C53D09DD","lastDispatchedFrom":"172.23.97.100:42898","lastDispatchedTo":"172.23.120.177:11210","requestId":6544,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001976","opaque":"0x22db","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":53834,"encodingMicros":8,"totalMicros":53931,"serverMicros":0}} | +--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:45:22,730 | test | DEBUG | MainProcess | pool-4-thread-4 | [task:batch_create:450] Reading values [u'test_docs-00001999', u'test_docs-00001998'] after failure 2020-01-13 22:45:22,732 | test | DEBUG | MainProcess | MainThread | [rest_client:_rebalance_status_and_progress:1486] Rebalance percentage: 0.00 % 2020-01-13 22:45:22,733 | test | INFO | MainProcess | pool-4-thread-4 | [table_view:display:72] Failed items: +--------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001999 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":36,"idempotent":false,"lastChannelId":"0000000000000024/0000000094F818B4","lastDispatchedFrom":"172.23.97.100:38446","lastDispatchedTo":"172.23.120.162:11210","requestId":6584,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001999","opaque":"0x2303","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":6201,"encodingMicros":6,"totalMicros":6340,"serverMicros":0}} | | test_docs-00001998 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":36,"idempotent":false,"lastChannelId":"0000000000000024/00000000C53D09DD","lastDispatchedFrom":"172.23.97.100:42898","lastDispatchedTo":"172.23.120.177:11210","requestId":6585,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001998","opaque":"0x2304","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":6717,"encodingMicros":6,"totalMicros":6804,"serverMicros":0}} | +--------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:45:22,743 | test | DEBUG | MainProcess | pool-4-thread-9 | [task:batch_create:450] Reading values [u'test_docs-00001211', u'test_docs-00001223', u'test_docs-00001212', u'test_docs-00001220', u'test_docs-00001210', u'test_docs-00001221', u'test_docs-00001215', u'test_docs-00001216', u'test_docs-00001213', u'test_docs-00001214', u'test_docs-00001208', u'test_docs-00001206', u'test_docs-00001217', u'test_docs-00001207'] after failure 2020-01-13 22:45:22,743 | test | DEBUG | MainProcess | pool-4-thread-7 | [task:batch_create:450] Reading values [u'test_docs-00001442', u'test_docs-00001454', u'test_docs-00001451', u'test_docs-00001436', u'test_docs-00001447', u'test_docs-00001439', u'test_docs-00001448', u'test_docs-00001438', u'test_docs-00001449'] after failure 2020-01-13 22:45:22,743 | test | DEBUG | MainProcess | pool-4-thread-10 | [task:batch_create:450] Reading values [u'test_docs-00001090', u'test_docs-00001093', u'test_docs-00001083', u'test_docs-00001080', u'test_docs-00001086', u'test_docs-00001084', u'test_docs-00001095', u'test_docs-00001085', u'test_docs-00001096', u'test_docs-00001089'] after failure 2020-01-13 22:45:22,743 | test | DEBUG | MainProcess | pool-4-thread-1 | [task:batch_create:450] Reading values [u'test_docs-00001344', u'test_docs-00001330', u'test_docs-00001341', u'test_docs-00001342', u'test_docs-00001348', u'test_docs-00001335', u'test_docs-00001338', u'test_docs-00001349', u'test_docs-00001339'] after failure 2020-01-13 22:45:23,614 | test | DEBUG | MainProcess | pool-4-thread-5 | [task:call:2870] Ops rate for 'default': 300.699301 2020-01-13 22:45:24,663 | test | DEBUG | MainProcess | pool-4-thread-5 | [task:call:2870] Ops rate for 'default': 372.627373 2020-01-13 22:45:25,361 | test | INFO | MainProcess | pool-4-thread-10 | [table_view:display:72] Failed items: +--------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001090 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":29,"idempotent":false,"lastChannelId":"000000000000001D/000000002DC44CA0","lastDispatchedFrom":"172.23.97.100:35330","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":6259,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001090","opaque":"0x21bc","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":13,"totalMicros":250040}} | | test_docs-00001093 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":29,"idempotent":false,"lastChannelId":"000000000000001D/000000003619BEF6","lastDispatchedFrom":"172.23.97.100:42790","lastDispatchedTo":"172.23.120.177:11210","requestId":6261,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001093","opaque":"0x21be","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":2818,"encodingMicros":5,"totalMicros":3793,"serverMicros":0}} | | test_docs-00001083 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":29,"idempotent":false,"lastChannelId":"000000000000001D/000000002DC44CA0","lastDispatchedFrom":"172.23.97.100:35330","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":6262,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001083","opaque":"0x21bf","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":5,"totalMicros":250009}} | | test_docs-00001080 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":29,"idempotent":false,"lastChannelId":"000000000000001D/000000002DC44CA0","lastDispatchedFrom":"172.23.97.100:35330","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":6264,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001080","opaque":"0x21c1","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":4,"totalMicros":249458}} | | test_docs-00001086 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":29,"idempotent":false,"lastChannelId":"000000000000001D/000000002DC44CA0","lastDispatchedFrom":"172.23.97.100:35330","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":6269,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001086","opaque":"0x21c6","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":5,"totalMicros":249655}} | | test_docs-00001084 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":29,"idempotent":false,"lastChannelId":"000000000000001D/00000000DE6CF467","lastDispatchedFrom":"172.23.97.100:38346","lastDispatchedTo":"172.23.120.162:11210","requestId":6277,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001084","opaque":"0x21ce","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":9451,"encodingMicros":5,"totalMicros":9778,"serverMicros":0}} | | test_docs-00001095 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":29,"idempotent":false,"lastChannelId":"000000000000001D/000000002DC44CA0","lastDispatchedFrom":"172.23.97.100:35330","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":6279,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001095","opaque":"0x21d0","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":6,"totalMicros":249139}} | | test_docs-00001085 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":29,"idempotent":false,"lastChannelId":"000000000000001D/000000002DC44CA0","lastDispatchedFrom":"172.23.97.100:35330","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":6281,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001085","opaque":"0x21d2","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":6,"totalMicros":249347}} | | test_docs-00001096 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":29,"idempotent":false,"lastChannelId":"000000000000001D/000000002DC44CA0","lastDispatchedFrom":"172.23.97.100:35330","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":6282,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001096","opaque":"0x21d3","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":4,"totalMicros":249415}} | | test_docs-00001089 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":29,"idempotent":false,"lastChannelId":"000000000000001D/000000002DC44CA0","lastDispatchedFrom":"172.23.97.100:35330","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":6288,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001089","opaque":"0x21d9","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":5,"totalMicros":249504}} | +--------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:45:25,364 | test | INFO | MainProcess | pool-4-thread-1 | [table_view:display:72] Failed items: +--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001344 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":31,"idempotent":false,"lastChannelId":"000000000000001F/00000000801AA50C","lastDispatchedFrom":"172.23.97.100:35358","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":6243,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001344","opaque":"0x21ac","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":4,"totalMicros":250701}} | | test_docs-00001330 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":31,"idempotent":false,"lastChannelId":"000000000000001F/00000000801AA50C","lastDispatchedFrom":"172.23.97.100:35358","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":6244,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001330","opaque":"0x21ad","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":5,"totalMicros":250775}} | | test_docs-00001341 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":31,"idempotent":false,"lastChannelId":"000000000000001F/00000000801AA50C","lastDispatchedFrom":"172.23.97.100:35358","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":6245,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001341","opaque":"0x21ae","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":4,"totalMicros":250846}} | | test_docs-00001342 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":31,"idempotent":false,"lastChannelId":"000000000000001F/000000009F2DDD97","lastDispatchedFrom":"172.23.97.100:38366","lastDispatchedTo":"172.23.120.162:11210","requestId":6247,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001342","opaque":"0x21b0","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":7892,"encodingMicros":4,"totalMicros":8001,"serverMicros":0}} | | test_docs-00001348 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":31,"idempotent":false,"lastChannelId":"000000000000001F/00000000C08192F6","lastDispatchedFrom":"172.23.97.100:42818","lastDispatchedTo":"172.23.120.177:11210","requestId":6251,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001348","opaque":"0x21b4","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":4303,"encodingMicros":4,"totalMicros":5639,"serverMicros":0}} | | test_docs-00001335 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":31,"idempotent":false,"lastChannelId":"000000000000001F/00000000801AA50C","lastDispatchedFrom":"172.23.97.100:35358","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":6254,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001335","opaque":"0x21b7","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":4,"totalMicros":250243}} | | test_docs-00001338 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":31,"idempotent":false,"lastChannelId":"000000000000001F/000000009F2DDD97","lastDispatchedFrom":"172.23.97.100:38366","lastDispatchedTo":"172.23.120.162:11210","requestId":6256,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001338","opaque":"0x21b9","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":10942,"encodingMicros":4,"totalMicros":12136,"serverMicros":0}} | | test_docs-00001349 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":31,"idempotent":false,"lastChannelId":"000000000000001F/000000009F2DDD97","lastDispatchedFrom":"172.23.97.100:38366","lastDispatchedTo":"172.23.120.162:11210","requestId":6257,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001349","opaque":"0x21ba","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":11036,"encodingMicros":4,"totalMicros":12201,"serverMicros":0}} | | test_docs-00001339 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":31,"idempotent":false,"lastChannelId":"000000000000001F/00000000801AA50C","lastDispatchedFrom":"172.23.97.100:35358","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":6258,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001339","opaque":"0x21bb","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":4,"totalMicros":250287}} | +--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:45:25,365 | test | INFO | MainProcess | pool-4-thread-9 | [table_view:display:72] Failed items: +--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001211 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":30,"idempotent":false,"lastChannelId":"000000000000001E/00000000246592B0","lastDispatchedFrom":"172.23.97.100:35344","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":6350,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001211","opaque":"0x2219","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":13,"totalMicros":243310}} | | test_docs-00001223 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":30,"idempotent":false,"lastChannelId":"000000000000001E/00000000246592B0","lastDispatchedFrom":"172.23.97.100:35344","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":6357,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001223","opaque":"0x2220","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":6,"totalMicros":242882}} | | test_docs-00001212 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":30,"idempotent":false,"lastChannelId":"000000000000001E/00000000246592B0","lastDispatchedFrom":"172.23.97.100:35344","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":6355,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001212","opaque":"0x221e","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":6,"totalMicros":243324}} | | test_docs-00001220 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":30,"idempotent":false,"lastChannelId":"000000000000001E/00000000246592B0","lastDispatchedFrom":"172.23.97.100:35344","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":6360,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001220","opaque":"0x2223","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":9,"totalMicros":242998}} | | test_docs-00001210 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":30,"idempotent":false,"lastChannelId":"000000000000001E/0000000091FD60CE","lastDispatchedFrom":"172.23.97.100:42806","lastDispatchedTo":"172.23.120.177:11210","requestId":6362,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001210","opaque":"0x2225","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":371,"encodingMicros":8,"totalMicros":531,"serverMicros":0}} | | test_docs-00001221 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":30,"idempotent":false,"lastChannelId":"000000000000001E/0000000091FD60CE","lastDispatchedFrom":"172.23.97.100:42806","lastDispatchedTo":"172.23.120.177:11210","requestId":6363,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001221","opaque":"0x2226","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":440,"encodingMicros":5,"totalMicros":562,"serverMicros":0}} | | test_docs-00001215 | com.couchbase.client.core.error.DurabilityAmbiguousException: The server returned with a durability ambiguous response on this request {"completed":true,"coreId":30,"idempotent":false,"lastChannelId":"000000000000001E/0000000091FD60CE","lastDispatchedFrom":"172.23.97.100:42806","lastDispatchedTo":"172.23.120.177:11210","requestId":6365,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001215","opaque":"0x2228","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"SYNC_WRITE_AMBIGUOUS","timeoutMs":5000,"timings":{"dispatchMicros":3246,"encodingMicros":8,"totalMicros":3366,"serverMicros":2}} | | test_docs-00001216 | com.couchbase.client.core.error.DurabilityAmbiguousException: The server returned with a durability ambiguous response on this request {"completed":true,"coreId":30,"idempotent":false,"lastChannelId":"000000000000001E/00000000DD1F2860","lastDispatchedFrom":"172.23.97.100:38350","lastDispatchedTo":"172.23.120.162:11210","requestId":6368,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001216","opaque":"0x222b","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"SYNC_WRITE_AMBIGUOUS","timeoutMs":5000,"timings":{"dispatchMicros":8209,"encodingMicros":13,"totalMicros":8332,"serverMicros":2}} | | test_docs-00001213 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":30,"idempotent":false,"lastChannelId":"000000000000001E/00000000DD1F2860","lastDispatchedFrom":"172.23.97.100:38350","lastDispatchedTo":"172.23.120.162:11210","requestId":6369,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001213","opaque":"0x222c","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":8249,"encodingMicros":9,"totalMicros":8304,"serverMicros":0}} | | test_docs-00001214 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":30,"idempotent":false,"lastChannelId":"000000000000001E/00000000246592B0","lastDispatchedFrom":"172.23.97.100:35344","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":6371,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001214","opaque":"0x222e","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":26,"totalMicros":242959}} | | test_docs-00001208 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":30,"idempotent":false,"lastChannelId":"000000000000001E/00000000246592B0","lastDispatchedFrom":"172.23.97.100:35344","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":6372,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001208","opaque":"0x222f","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":10,"totalMicros":242495}} | | test_docs-00001206 | com.couchbase.client.core.error.DurabilityAmbiguousException: The server returned with a durability ambiguous response on this request {"completed":true,"coreId":30,"idempotent":false,"lastChannelId":"000000000000001E/0000000091FD60CE","lastDispatchedFrom":"172.23.97.100:42806","lastDispatchedTo":"172.23.120.177:11210","requestId":6375,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001206","opaque":"0x2232","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"SYNC_WRITE_AMBIGUOUS","timeoutMs":5000,"timings":{"dispatchMicros":5913,"encodingMicros":7,"totalMicros":6212,"serverMicros":1}} | | test_docs-00001217 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":30,"idempotent":false,"lastChannelId":"000000000000001E/0000000091FD60CE","lastDispatchedFrom":"172.23.97.100:42806","lastDispatchedTo":"172.23.120.177:11210","requestId":6376,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001217","opaque":"0x2233","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":6012,"encodingMicros":12,"totalMicros":6232,"serverMicros":0}} | | test_docs-00001207 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":30,"idempotent":false,"lastChannelId":"000000000000001E/00000000246592B0","lastDispatchedFrom":"172.23.97.100:35344","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":6377,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001207","opaque":"0x2234","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":6,"totalMicros":242508}} | +--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:45:25,368 | test | INFO | MainProcess | pool-4-thread-7 | [table_view:display:72] Failed items: +--------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001442 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":32,"idempotent":false,"lastChannelId":"0000000000000020/000000006C5901F1","lastDispatchedFrom":"172.23.97.100:35370","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":6320,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001442","opaque":"0x21fb","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":6,"totalMicros":244144}} | | test_docs-00001454 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":32,"idempotent":false,"lastChannelId":"0000000000000020/000000006C5901F1","lastDispatchedFrom":"172.23.97.100:35370","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":6323,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001454","opaque":"0x21fe","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":4,"totalMicros":244216}} | | test_docs-00001451 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":32,"idempotent":false,"lastChannelId":"0000000000000020/000000006C5901F1","lastDispatchedFrom":"172.23.97.100:35370","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":6325,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001451","opaque":"0x2200","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":4,"totalMicros":243682}} | | test_docs-00001436 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":32,"idempotent":false,"lastChannelId":"0000000000000020/000000006C5901F1","lastDispatchedFrom":"172.23.97.100:35370","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":6330,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001436","opaque":"0x2205","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":4,"totalMicros":243711}} | | test_docs-00001447 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":32,"idempotent":false,"lastChannelId":"0000000000000020/000000006C5901F1","lastDispatchedFrom":"172.23.97.100:35370","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":6331,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001447","opaque":"0x2206","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":4,"totalMicros":243768}} | | test_docs-00001439 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":32,"idempotent":false,"lastChannelId":"0000000000000020/000000006C5901F1","lastDispatchedFrom":"172.23.97.100:35370","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":6334,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001439","opaque":"0x2209","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":5,"totalMicros":243750}} | | test_docs-00001448 | com.couchbase.client.core.error.RequestCanceledException: InsertRequest {"cancelled":true,"completed":true,"coreId":32,"idempotent":false,"lastChannelId":"0000000000000020/000000006C5901F1","lastDispatchedFrom":"172.23.97.100:35370","lastDispatchedTo":"172.23.120.19:11210","reason":"NO_MORE_RETRIES (CHANNEL_CLOSED_WHILE_IN_FLIGHT)","requestId":6336,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001448","opaque":"0x220b","scope":"_default","syncDurability":{"present":true},"type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":4,"totalMicros":243878}} | | test_docs-00001438 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":32,"idempotent":false,"lastChannelId":"0000000000000020/00000000E7D92B21","lastDispatchedFrom":"172.23.97.100:42832","lastDispatchedTo":"172.23.120.177:11210","requestId":6337,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001438","opaque":"0x220c","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":6698,"encodingMicros":4,"totalMicros":8311,"serverMicros":0}} | | test_docs-00001449 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":32,"idempotent":false,"lastChannelId":"0000000000000020/00000000E7D92B21","lastDispatchedFrom":"172.23.97.100:42832","lastDispatchedTo":"172.23.120.177:11210","requestId":6338,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001449","opaque":"0x220d","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":6802,"encodingMicros":4,"totalMicros":8352,"serverMicros":0}} | +--------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:45:25,411 | test | DEBUG | MainProcess | pool-4-thread-9 | [task:batch_create:450] Reading values [u'test_docs-00001240', u'test_docs-00001233', u'test_docs-00001242', u'test_docs-00001243', u'test_docs-00001237', u'test_docs-00001225', u'test_docs-00001236'] after failure 2020-01-13 22:45:25,414 | test | INFO | MainProcess | pool-4-thread-9 | [table_view:display:72] Failed items: +--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001240 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":30,"idempotent":false,"lastChannelId":"000000000000001E/00000000DD1F2860","lastDispatchedFrom":"172.23.97.100:38350","lastDispatchedTo":"172.23.120.162:11210","requestId":6734,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001240","opaque":"0x2441","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":1858,"encodingMicros":16,"totalMicros":2103,"serverMicros":0}} | | test_docs-00001233 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":30,"idempotent":false,"lastChannelId":"000000000000001E/00000000DD1F2860","lastDispatchedFrom":"172.23.97.100:38350","lastDispatchedTo":"172.23.120.162:11210","requestId":6737,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001233","opaque":"0x2444","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":9440,"encodingMicros":4,"totalMicros":9574,"serverMicros":0}} | | test_docs-00001242 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":30,"idempotent":false,"lastChannelId":"000000000000001E/0000000091FD60CE","lastDispatchedFrom":"172.23.97.100:42806","lastDispatchedTo":"172.23.120.177:11210","requestId":6741,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001242","opaque":"0x2448","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":12189,"encodingMicros":4,"totalMicros":12454,"serverMicros":0}} | | test_docs-00001243 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":30,"idempotent":false,"lastChannelId":"000000000000001E/0000000091FD60CE","lastDispatchedFrom":"172.23.97.100:42806","lastDispatchedTo":"172.23.120.177:11210","requestId":6743,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001243","opaque":"0x244a","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":14085,"encodingMicros":4,"totalMicros":14331,"serverMicros":0}} | | test_docs-00001237 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":30,"idempotent":false,"lastChannelId":"000000000000001E/0000000091FD60CE","lastDispatchedFrom":"172.23.97.100:42806","lastDispatchedTo":"172.23.120.177:11210","requestId":6745,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001237","opaque":"0x244c","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":14138,"encodingMicros":4,"totalMicros":14344,"serverMicros":0}} | | test_docs-00001225 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":30,"idempotent":false,"lastChannelId":"000000000000001E/00000000DD1F2860","lastDispatchedFrom":"172.23.97.100:38350","lastDispatchedTo":"172.23.120.162:11210","requestId":6749,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001225","opaque":"0x2450","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":24577,"encodingMicros":3,"totalMicros":24692,"serverMicros":0}} | | test_docs-00001236 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":30,"idempotent":false,"lastChannelId":"000000000000001E/0000000091FD60CE","lastDispatchedFrom":"172.23.97.100:42806","lastDispatchedTo":"172.23.120.177:11210","requestId":6750,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001236","opaque":"0x2451","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":30373,"encodingMicros":4,"totalMicros":30542,"serverMicros":0}} | +--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:45:25,421 | test | DEBUG | MainProcess | pool-4-thread-7 | [task:batch_create:450] Reading values [u'test_docs-00001460', u'test_docs-00001465', u'test_docs-00001473', u'test_docs-00001468'] after failure 2020-01-13 22:45:25,421 | test | DEBUG | MainProcess | pool-4-thread-1 | [task:batch_create:450] Reading values [u'test_docs-00001352', u'test_docs-00001358', u'test_docs-00001369', u'test_docs-00001359', u'test_docs-00001357'] after failure 2020-01-13 22:45:25,423 | test | INFO | MainProcess | pool-4-thread-7 | [table_view:display:72] Failed items: +--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001460 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":32,"idempotent":false,"lastChannelId":"0000000000000020/000000006028A8F6","lastDispatchedFrom":"172.23.97.100:38394","lastDispatchedTo":"172.23.120.162:11210","requestId":6754,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001460","opaque":"0x2455","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":491,"encodingMicros":18,"totalMicros":692,"serverMicros":0}} | | test_docs-00001465 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":32,"idempotent":false,"lastChannelId":"0000000000000020/000000006028A8F6","lastDispatchedFrom":"172.23.97.100:38394","lastDispatchedTo":"172.23.120.162:11210","requestId":6760,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001465","opaque":"0x245b","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":9120,"encodingMicros":7,"totalMicros":9210,"serverMicros":0}} | | test_docs-00001473 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":32,"idempotent":false,"lastChannelId":"0000000000000020/000000006028A8F6","lastDispatchedFrom":"172.23.97.100:38394","lastDispatchedTo":"172.23.120.162:11210","requestId":6762,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001473","opaque":"0x245d","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":10884,"encodingMicros":6,"totalMicros":10969,"serverMicros":0}} | | test_docs-00001468 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":32,"idempotent":false,"lastChannelId":"0000000000000020/000000006028A8F6","lastDispatchedFrom":"172.23.97.100:38394","lastDispatchedTo":"172.23.120.162:11210","requestId":6766,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001468","opaque":"0x2461","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":19952,"encodingMicros":8,"totalMicros":20030,"serverMicros":0}} | +--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:45:25,423 | test | INFO | MainProcess | pool-4-thread-1 | [table_view:display:72] Failed items: +--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001352 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":31,"idempotent":false,"lastChannelId":"000000000000001F/000000009F2DDD97","lastDispatchedFrom":"172.23.97.100:38366","lastDispatchedTo":"172.23.120.162:11210","requestId":6723,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001352","opaque":"0x2436","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":35786,"encodingMicros":3,"totalMicros":35905,"serverMicros":0}} | | test_docs-00001358 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":31,"idempotent":false,"lastChannelId":"000000000000001F/00000000C08192F6","lastDispatchedFrom":"172.23.97.100:42818","lastDispatchedTo":"172.23.120.177:11210","requestId":6727,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001358","opaque":"0x243a","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":19218,"encodingMicros":4,"totalMicros":19839,"serverMicros":0}} | | test_docs-00001369 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":31,"idempotent":false,"lastChannelId":"000000000000001F/000000009F2DDD97","lastDispatchedFrom":"172.23.97.100:38366","lastDispatchedTo":"172.23.120.162:11210","requestId":6728,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001369","opaque":"0x243b","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":37861,"encodingMicros":4,"totalMicros":37940,"serverMicros":0}} | | test_docs-00001359 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":31,"idempotent":false,"lastChannelId":"000000000000001F/00000000C08192F6","lastDispatchedFrom":"172.23.97.100:42818","lastDispatchedTo":"172.23.120.177:11210","requestId":6729,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001359","opaque":"0x243c","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":19284,"encodingMicros":4,"totalMicros":19865,"serverMicros":0}} | | test_docs-00001357 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":31,"idempotent":false,"lastChannelId":"000000000000001F/000000009F2DDD97","lastDispatchedFrom":"172.23.97.100:38366","lastDispatchedTo":"172.23.120.162:11210","requestId":6732,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001357","opaque":"0x243f","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":37884,"encodingMicros":3,"totalMicros":37939,"serverMicros":0}} | +--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:45:25,434 | test | DEBUG | MainProcess | pool-4-thread-9 | [task:batch_create:450] Reading values [u'test_docs-00001245', u'test_docs-00001246', u'test_docs-00001247'] after failure 2020-01-13 22:45:25,437 | test | INFO | MainProcess | pool-4-thread-9 | [table_view:display:72] Failed items: +--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001245 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":30,"idempotent":false,"lastChannelId":"000000000000001E/00000000DD1F2860","lastDispatchedFrom":"172.23.97.100:38350","lastDispatchedTo":"172.23.120.162:11210","requestId":6789,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001245","opaque":"0x2478","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":1135,"encodingMicros":26,"totalMicros":1262,"serverMicros":0}} | | test_docs-00001246 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":30,"idempotent":false,"lastChannelId":"000000000000001E/0000000091FD60CE","lastDispatchedFrom":"172.23.97.100:42806","lastDispatchedTo":"172.23.120.177:11210","requestId":6792,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001246","opaque":"0x247b","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":11633,"encodingMicros":11,"totalMicros":11730,"serverMicros":0}} | | test_docs-00001247 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":30,"idempotent":false,"lastChannelId":"000000000000001E/0000000091FD60CE","lastDispatchedFrom":"172.23.97.100:42806","lastDispatchedTo":"172.23.120.177:11210","requestId":6793,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001247","opaque":"0x247c","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":11647,"encodingMicros":10,"totalMicros":11735,"serverMicros":0}} | +--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:45:25,443 | test | DEBUG | MainProcess | pool-4-thread-10 | [task:batch_create:450] Reading values [u'test_docs-00001102', u'test_docs-00001113', u'test_docs-00001100', u'test_docs-00001116', u'test_docs-00001105', u'test_docs-00001114'] after failure 2020-01-13 22:45:25,446 | test | INFO | MainProcess | pool-4-thread-10 | [table_view:display:72] Failed items: +--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001102 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":29,"idempotent":false,"lastChannelId":"000000000000001D/00000000DE6CF467","lastDispatchedFrom":"172.23.97.100:38346","lastDispatchedTo":"172.23.120.162:11210","requestId":6696,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001102","opaque":"0x241b","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":3503,"encodingMicros":6,"totalMicros":3985,"serverMicros":0}} | | test_docs-00001113 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":29,"idempotent":false,"lastChannelId":"000000000000001D/00000000DE6CF467","lastDispatchedFrom":"172.23.97.100:38346","lastDispatchedTo":"172.23.120.162:11210","requestId":6697,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001113","opaque":"0x241c","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":3573,"encodingMicros":7,"totalMicros":4010,"serverMicros":0}} | | test_docs-00001100 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":29,"idempotent":false,"lastChannelId":"000000000000001D/00000000DE6CF467","lastDispatchedFrom":"172.23.97.100:38346","lastDispatchedTo":"172.23.120.162:11210","requestId":6699,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001100","opaque":"0x241e","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":10978,"encodingMicros":5,"totalMicros":11409,"serverMicros":0}} | | test_docs-00001116 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":29,"idempotent":false,"lastChannelId":"000000000000001D/000000003619BEF6","lastDispatchedFrom":"172.23.97.100:42790","lastDispatchedTo":"172.23.120.177:11210","requestId":6702,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001116","opaque":"0x2421","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":14091,"encodingMicros":5,"totalMicros":14387,"serverMicros":0}} | | test_docs-00001105 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":29,"idempotent":false,"lastChannelId":"000000000000001D/00000000DE6CF467","lastDispatchedFrom":"172.23.97.100:38346","lastDispatchedTo":"172.23.120.162:11210","requestId":6701,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001105","opaque":"0x2420","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":15169,"encodingMicros":5,"totalMicros":15557,"serverMicros":0}} | | test_docs-00001114 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":29,"idempotent":false,"lastChannelId":"000000000000001D/00000000DE6CF467","lastDispatchedFrom":"172.23.97.100:38346","lastDispatchedTo":"172.23.120.162:11210","requestId":6706,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001114","opaque":"0x2425","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":17271,"encodingMicros":5,"totalMicros":17592,"serverMicros":0}} | +--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:45:25,461 | test | DEBUG | MainProcess | pool-4-thread-10 | [task:batch_create:450] Reading values [u'test_docs-00001120', u'test_docs-00001123', u'test_docs-00001122'] after failure 2020-01-13 22:45:25,463 | test | INFO | MainProcess | pool-4-thread-10 | [table_view:display:72] Failed items: +--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001120 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":29,"idempotent":false,"lastChannelId":"000000000000001D/00000000DE6CF467","lastDispatchedFrom":"172.23.97.100:38346","lastDispatchedTo":"172.23.120.162:11210","requestId":6841,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001120","opaque":"0x24ac","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":1319,"encodingMicros":16,"totalMicros":1434,"serverMicros":0}} | | test_docs-00001123 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":29,"idempotent":false,"lastChannelId":"000000000000001D/000000003619BEF6","lastDispatchedFrom":"172.23.97.100:42790","lastDispatchedTo":"172.23.120.177:11210","requestId":6842,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001123","opaque":"0x24ad","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":351,"encodingMicros":8,"totalMicros":459,"serverMicros":0}} | | test_docs-00001122 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":29,"idempotent":false,"lastChannelId":"000000000000001D/000000003619BEF6","lastDispatchedFrom":"172.23.97.100:42790","lastDispatchedTo":"172.23.120.177:11210","requestId":6845,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001122","opaque":"0x24b0","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":9059,"encodingMicros":6,"totalMicros":9126,"serverMicros":0}} | +--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:45:25,463 | test | ERROR | MainProcess | pool-4-thread-6 | [task:call:1483] Failed to load 21 docs from 1000 to 1125 2020-01-13 22:45:25,463 | test | ERROR | MainProcess | pool-4-thread-6 | [task:call:1483] Failed to load 29 docs from 1125 to 1250 2020-01-13 22:45:25,464 | test | ERROR | MainProcess | pool-4-thread-6 | [task:call:1483] Failed to load 15 docs from 1250 to 1375 2020-01-13 22:45:25,469 | test | DEBUG | MainProcess | pool-4-thread-7 | [task:batch_create:450] Reading values [u'test_docs-00001482', u'test_docs-00001494', u'test_docs-00001491', u'test_docs-00001486', u'test_docs-00001487', u'test_docs-00001476', u'test_docs-00001485', u'test_docs-00001478'] after failure 2020-01-13 22:45:25,473 | test | INFO | MainProcess | pool-4-thread-7 | [table_view:display:72] Failed items: +--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Create doc_Id | Exception | +--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test_docs-00001482 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":32,"idempotent":false,"lastChannelId":"0000000000000020/000000006028A8F6","lastDispatchedFrom":"172.23.97.100:38394","lastDispatchedTo":"172.23.120.162:11210","requestId":6809,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001482","opaque":"0x248c","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":359,"encodingMicros":6,"totalMicros":466,"serverMicros":0}} | | test_docs-00001494 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":32,"idempotent":false,"lastChannelId":"0000000000000020/000000006028A8F6","lastDispatchedFrom":"172.23.97.100:38394","lastDispatchedTo":"172.23.120.162:11210","requestId":6812,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001494","opaque":"0x248f","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":12369,"encodingMicros":5,"totalMicros":12461,"serverMicros":0}} | | test_docs-00001491 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":32,"idempotent":false,"lastChannelId":"0000000000000020/00000000E7D92B21","lastDispatchedFrom":"172.23.97.100:42832","lastDispatchedTo":"172.23.120.177:11210","requestId":6814,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001491","opaque":"0x2491","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":19400,"encodingMicros":4,"totalMicros":19474,"serverMicros":0}} | | test_docs-00001486 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":32,"idempotent":false,"lastChannelId":"0000000000000020/00000000E7D92B21","lastDispatchedFrom":"172.23.97.100:42832","lastDispatchedTo":"172.23.120.177:11210","requestId":6818,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001486","opaque":"0x2495","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":28560,"encodingMicros":4,"totalMicros":28640,"serverMicros":0}} | | test_docs-00001487 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":32,"idempotent":false,"lastChannelId":"0000000000000020/00000000E7D92B21","lastDispatchedFrom":"172.23.97.100:42832","lastDispatchedTo":"172.23.120.177:11210","requestId":6820,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001487","opaque":"0x2497","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":28590,"encodingMicros":4,"totalMicros":28674,"serverMicros":0}} | | test_docs-00001476 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":32,"idempotent":false,"lastChannelId":"0000000000000020/000000006028A8F6","lastDispatchedFrom":"172.23.97.100:38394","lastDispatchedTo":"172.23.120.162:11210","requestId":6819,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001476","opaque":"0x2496","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":30808,"encodingMicros":4,"totalMicros":30895,"serverMicros":0}} | | test_docs-00001485 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":32,"idempotent":false,"lastChannelId":"0000000000000020/000000006028A8F6","lastDispatchedFrom":"172.23.97.100:38394","lastDispatchedTo":"172.23.120.162:11210","requestId":6822,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001485","opaque":"0x2499","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":32910,"encodingMicros":5,"totalMicros":32983,"serverMicros":0}} | | test_docs-00001478 | com.couchbase.client.core.error.DurabilityImpossibleException: With the current cluster configuration, the requested durability guarantees are impossible {"completed":true,"coreId":32,"idempotent":false,"lastChannelId":"0000000000000020/00000000E7D92B21","lastDispatchedFrom":"172.23.97.100:42832","lastDispatchedTo":"172.23.120.177:11210","requestId":6826,"requestType":"InsertRequest","retried":0,"service":{"bucket":"default","collection":"_default","documentId":"test_docs-00001478","opaque":"0x249d","scope":"_default","syncDurability":{"present":true},"type":"kv"},"status":"DURABILITY_IMPOSSIBLE","timeoutMs":5000,"timings":{"dispatchMicros":35771,"encodingMicros":4,"totalMicros":35842,"serverMicros":0}} | +--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-01-13 22:45:25,486 | test | ERROR | MainProcess | pool-4-thread-6 | [task:call:1483] Failed to load 22 docs from 1375 to 1500 2020-01-13 22:45:25,487 | test | ERROR | MainProcess | pool-4-thread-6 | [task:call:1483] Failed to load 28 docs from 1500 to 1625 2020-01-13 22:45:25,487 | test | ERROR | MainProcess | pool-4-thread-6 | [task:call:1483] Failed to load 19 docs from 1625 to 1750 2020-01-13 22:45:25,487 | test | ERROR | MainProcess | pool-4-thread-6 | [task:call:1483] Failed to load 17 docs from 1750 to 1875 2020-01-13 22:45:25,489 | test | ERROR | MainProcess | pool-4-thread-6 | [task:call:1483] Failed to load 38 docs from 1875 to 2000 2020-01-13 22:45:25,664 | test | INFO | MainProcess | pool-4-thread-5 | [table_view:display:72] Ops Trend for bucket 'default' +-------+----------------------------------------------------------------------------------------------------------+---------+ | Min | Trend | Max | +-------+----------------------------------------------------------------------------------------------------------+---------+ | 0.000 | **................................................................................*....................X | 372.627 | +-------+----------------------------------------------------------------------------------------------------------+---------+ 2020-01-13 22:45:25,664 | test | DEBUG | MainProcess | pool-4-thread-6 | [sdk_client3:close:143] Closing down the cluster 2020-01-13 22:45:25,691 | test | DEBUG | MainProcess | pool-4-thread-6 | [sdk_client3:close:147] Cluster disconnected and env shutdown 2020-01-13 22:45:25,691 | test | DEBUG | MainProcess | pool-4-thread-6 | [sdk_client3:close:143] Closing down the cluster 2020-01-13 22:45:25,717 | test | DEBUG | MainProcess | pool-4-thread-6 | [sdk_client3:close:147] Cluster disconnected and env shutdown 2020-01-13 22:45:25,717 | test | DEBUG | MainProcess | pool-4-thread-6 | [sdk_client3:close:143] Closing down the cluster 2020-01-13 22:45:25,740 | test | DEBUG | MainProcess | pool-4-thread-6 | [sdk_client3:close:147] Cluster disconnected and env shutdown 2020-01-13 22:45:25,742 | test | DEBUG | MainProcess | pool-4-thread-6 | [sdk_client3:close:143] Closing down the cluster 2020-01-13 22:45:25,766 | test | DEBUG | MainProcess | pool-4-thread-6 | [sdk_client3:close:147] Cluster disconnected and env shutdown 2020-01-13 22:45:25,766 | test | DEBUG | MainProcess | pool-4-thread-6 | [sdk_client3:close:143] Closing down the cluster 2020-01-13 22:45:25,792 | test | DEBUG | MainProcess | pool-4-thread-6 | [sdk_client3:close:147] Cluster disconnected and env shutdown 2020-01-13 22:45:25,792 | test | DEBUG | MainProcess | pool-4-thread-6 | [sdk_client3:close:143] Closing down the cluster 2020-01-13 22:45:25,815 | test | DEBUG | MainProcess | pool-4-thread-6 | [sdk_client3:close:147] Cluster disconnected and env shutdown 2020-01-13 22:45:25,816 | test | DEBUG | MainProcess | pool-4-thread-6 | [sdk_client3:close:143] Closing down the cluster 2020-01-13 22:45:25,841 | test | DEBUG | MainProcess | pool-4-thread-6 | [sdk_client3:close:147] Cluster disconnected and env shutdown 2020-01-13 22:45:25,841 | test | DEBUG | MainProcess | pool-4-thread-6 | [sdk_client3:close:143] Closing down the cluster 2020-01-13 22:45:25,865 | test | DEBUG | MainProcess | pool-4-thread-6 | [sdk_client3:close:147] Cluster disconnected and env shutdown 2020-01-13 22:45:47,760 | test | INFO | MainProcess | MainThread | [rest_client:monitorRebalance:1423] Rebalance done. Taken 25.0469999313 seconds to complete 2020-01-13 22:45:47,762 | test | INFO | MainProcess | MainThread | [rest_client:monitorRebalance:1425] Sleep for 5 seconds after rebalance 2020-01-13 22:45:52,776 | test | DEBUG | MainProcess | MainThread | [rest_client:add_node:1093] Adding remote node 172.23.120.192:8091 to cluster 172.23.120.162:8091 2020-01-13 22:45:54,588 | test | DEBUG | MainProcess | MainThread | [rest_client:add_node:1093] Adding remote node 172.23.120.19:8091 to cluster 172.23.120.162:8091 2020-01-13 22:45:59,292 | test | DEBUG | MainProcess | MainThread | [rest_client:rebalance:1327] Rebalance params: {'password': 'password', 'knownNodes': u'ns_1@172.23.120.192,ns_1@172.23.120.19,ns_1@172.23.120.162,ns_1@172.23.120.177', 'ejectedNodes': '', 'user': 'Administrator'} 2020-01-13 22:45:59,303 | test | DEBUG | MainProcess | MainThread | [rest_client:rebalance:1332] Rebalance operation started 2020-01-13 22:45:59,312 | test | DEBUG | MainProcess | MainThread | [rest_client:_rebalance_status_and_progress:1486] Rebalance percentage: 0.00 % 2020-01-13 22:46:09,322 | test | DEBUG | MainProcess | MainThread | [rest_client:_rebalance_status_and_progress:1486] Rebalance percentage: 11.13 % 2020-01-13 22:46:19,338 | test | DEBUG | MainProcess | MainThread | [rest_client:_rebalance_status_and_progress:1486] Rebalance percentage: 40.58 % 2020-01-13 22:46:29,354 | test | DEBUG | MainProcess | MainThread | [rest_client:_rebalance_status_and_progress:1486] Rebalance percentage: 65.36 % 2020-01-13 22:46:39,368 | test | DEBUG | MainProcess | MainThread | [rest_client:_rebalance_status_and_progress:1486] Rebalance percentage: 83.05 % 2020-01-13 22:47:04,395 | test | INFO | MainProcess | MainThread | [rest_client:monitorRebalance:1423] Rebalance done. Taken 65.0920000076 seconds to complete 2020-01-13 22:47:04,397 | test | INFO | MainProcess | MainThread | [rest_client:monitorRebalance:1425] Sleep for 5 seconds after rebalance 2020-01-13 22:47:09,408 | test | DEBUG | MainProcess | MainThread | [sdk_client3:_createConn:96] Creating cluster connection 2020-01-13 22:47:09,846 | test | DEBUG | MainProcess | MainThread | [sdk_client3:close:143] Closing down the cluster 2020-01-13 22:47:09,871 | test | DEBUG | MainProcess | MainThread | [sdk_client3:close:147] Cluster disconnected and env shutdown 2020-01-13 22:47:09,872 | test | DEBUG | MainProcess | MainThread | [bucket_ready_functions:log_doc_ops_task_failures:747] Docs succeeded for expected retries for 'create' (189): [u'test_docs-00001708', u'test_docs-00001706', u'test_docs-00001947', u'test_docs-00001705', u'test_docs-00001709', u'test_docs-00001942', u'test_docs-00001941', u'test_docs-00001825', u'test_docs-00001703', u'test_docs-00001426', u'test_docs-00001944', u'test_docs-00001943', u'test_docs-00001719', u'test_docs-00001718', u'test_docs-00001839', u'test_docs-00001436', u'test_docs-00001710', u'test_docs-00001952', u'test_docs-00001830', u'test_docs-00001951', u'test_docs-00001439', u'test_docs-00001715', u'test_docs-00001957', u'test_docs-00001835', u'test_docs-00001713', u'test_docs-00001955', u'test_docs-00001438', u'test_docs-00001833', u'test_docs-00001954', u'test_docs-00001609', u'test_docs-00001607', u'test_docs-00001728', u'test_docs-00001606', u'test_docs-00001848', u'test_docs-00001969', u'test_docs-00001442', u'test_docs-00001960', u'test_docs-00001201', u'test_docs-00001325', u'test_docs-00001204', u'test_docs-00001601', u'test_docs-00001964', u'test_docs-00001447', u'test_docs-00001600', u'test_docs-00001963', u'test_docs-00001962', u'test_docs-00001565', u'test_docs-00001961', u'test_docs-00001208', u'test_docs-00001605', u'test_docs-00001604', u'test_docs-00001725', u'test_docs-00001967', u'test_docs-00001206', u'test_docs-00001448', u'test_docs-00001603', u'test_docs-00001966', u'test_docs-00001207', u'test_docs-00001449', u'test_docs-00001602', u'test_docs-00001965', u'test_docs-00001617', u'test_docs-00001738', u'test_docs-00001859', u'test_docs-00001211', u'test_docs-00001971', u'test_docs-00001212', u'test_docs-00001454', u'test_docs-00001970', u'test_docs-00001330', u'test_docs-00001451', u'test_docs-00001210', u'test_docs-00001693', u'test_docs-00001215', u'test_docs-00001579', u'test_docs-00001612', u'test_docs-00001975', u'test_docs-00001216', u'test_docs-00001611', u'test_docs-00001699', u'test_docs-00001974', u'test_docs-00001213', u'test_docs-00001610', u'test_docs-00001973', u'test_docs-00001214', u'test_docs-00001335', u'test_docs-00001576', u'test_docs-00001972', u'test_docs-00001616', u'test_docs-00001858', u'test_docs-00001615', u'test_docs-00001217', u'test_docs-00001338', u'test_docs-00001614', u'test_docs-00001977', u'test_docs-00001339', u'test_docs-00001613', u'test_docs-00001976', u'test_docs-00001749', u'test_docs-00001460', u'test_docs-00001186', u'test_docs-00001068', u'test_docs-00001189', u'test_docs-00001740', u'test_docs-00001102', u'test_docs-00001223', u'test_docs-00001344', u'test_docs-00001465', u'test_docs-00001220', u'test_docs-00001341', u'test_docs-00001100', u'test_docs-00001067', u'test_docs-00001221', u'test_docs-00001342', u'test_docs-00001105', u'test_docs-00001468', u'test_docs-00001623', u'test_docs-00001865', u'test_docs-00001348', u'test_docs-00001589', u'test_docs-00001622', u'test_docs-00001743', u'test_docs-00001588', u'test_docs-00001621', u'test_docs-00001863', u'test_docs-00001225', u'test_docs-00001620', u'test_docs-00001748', u'test_docs-00001869', u'test_docs-00001868', u'test_docs-00001349', u'test_docs-00001988', u'test_docs-00001624', u'test_docs-00001866', u'test_docs-00001918', u'test_docs-00001083', u'test_docs-00001080', u'test_docs-00001195', u'test_docs-00001233', u'test_docs-00001113', u'test_docs-00001476', u'test_docs-00001352', u'test_docs-00001473', u'test_docs-00001870', u'test_docs-00001116', u'test_docs-00001237', u'test_docs-00001358', u'test_docs-00001359', u'test_docs-00001114', u'test_docs-00001236', u'test_docs-00001357', u'test_docs-00001478', u'test_docs-00001598', u'test_docs-00001873', u'test_docs-00001999', u'test_docs-00001998', u'test_docs-00001929', u'test_docs-00001090', u'test_docs-00001093', u'test_docs-00001086', u'test_docs-00001240', u'test_docs-00001482', u'test_docs-00001120', u'test_docs-00001084', u'test_docs-00001085', u'test_docs-00001123', u'test_docs-00001486', u'test_docs-00001245', u'test_docs-00001487', u'test_docs-00001242', u'test_docs-00001089', u'test_docs-00001122', u'test_docs-00001243', u'test_docs-00001485', u'test_docs-00001369', u'test_docs-00001246', u'test_docs-00001247', u'test_docs-00001923', u'test_docs-00001939', u'test_docs-00001937', u'test_docs-00001936', u'test_docs-00001494', u'test_docs-00001095', u'test_docs-00001491', u'test_docs-00001096', u'test_docs-00001930', u'test_docs-00001814', u'test_docs-00001935', u'test_docs-00001811'] 2020-01-13 22:47:09,874 | test | INFO | MainProcess | MainThread | [swaprebalancetests:validate_docs:116] Validating docs 2020-01-13 22:47:09,887 | test | DEBUG | MainProcess | MainThread | [sdk_client3:_createConn:96] Creating cluster connection 2020-01-13 22:47:10,045 | test | DEBUG | MainProcess | pool-4-thread-2 | [sdk_client3:close:143] Closing down the cluster 2020-01-13 22:47:10,069 | test | DEBUG | MainProcess | pool-4-thread-2 | [sdk_client3:close:147] Cluster disconnected and env shutdown 2020-01-13 22:47:10,072 | test | DEBUG | MainProcess | MainThread | [sdk_client3:close:143] Closing down the cluster 2020-01-13 22:47:10,102 | test | INFO | MainProcess | MainThread | [table_view:display:72] Cluster statistics +----------------+-----------------+-----------------+------------+------------+---------------------+-------------------+-----------------------+ | Node | Services | CPU_utilization | Mem_total | Mem_free | Swap_mem_used | Active / Replica | Version | +----------------+-----------------+-----------------+------------+------------+---------------------+-------------------+-----------------------+ | 172.23.120.192 | kv | 2.04081632653 | 3965001728 | 3232808960 | 0 / 3758092288 | 439 / 874 | 6.5.0-4960-enterprise | | 172.23.120.19 | kv | 3.26633165829 | 4220141568 | 3532271616 | 262144 / 3758092288 | 426 / 892 | 6.5.0-4960-enterprise | | 172.23.120.162 | index, kv, n1ql | 1.75879396985 | 3963588608 | 2989252608 | 0 / 2147479552 | 474 / 933 | 6.5.0-4960-enterprise | | 172.23.120.177 | kv | 1.00502512563 | 3965001728 | 3180670976 | 0 / 3758092288 | 472 / 923 | 6.5.0-4960-enterprise | +----------------+-----------------+-----------------+------------+------------+---------------------+-------------------+-----------------------+ 2020-01-13 22:47:10,115 | test | CRITICAL | MainProcess | MainThread | [basetestcase:tearDownEverything:344] Skipping get_trace !! 2020-01-13 22:47:10,138 | test | DEBUG | MainProcess | MainThread | [basetestcase:tearDownEverything:350] Cleaning up cluster 2020-01-13 22:47:10,200 | test | DEBUG | MainProcess | MainThread | [bucket_ready_functions:parse_get_bucket_json:2064] read 1024 vbuckets 2020-01-13 22:47:10,200 | test | DEBUG | MainProcess | MainThread | [bucket_ready_functions:parse_get_bucket_json:2067] Stats: {u'opsPerSec': 0, u'diskFetches': 0, u'dataUsed': 59416032, u'itemCount': 1811, u'quotaPercentUsed': 1.680236384660915, u'storageTotals': {u'hdd': {u'usedByData': 74010680, u'used': 39351700929L, u'quotaTotal': 144901136384L, u'total': 144901136384L, u'free': 75874132132L}, u'ram': {u'usedByData': 94646880, u'used': 13537705984L, u'quotaTotal': 8451522560L, u'total': 16113733632L, u'quotaUsed': 5632950272L, u'quotaUsedPerNode': 1408237568, u'quotaTotalPerNode': 2112880640}}, u'diskUsed': 74010680, u'vbActiveNumNonResident': 0, u'memUsed': 94646880} 2020-01-13 22:47:10,203 | test | DEBUG | MainProcess | MainThread | [bucket_ready_functions:delete_all_buckets:246] Deleting existing buckets [u'default'] on 172.23.120.162 2020-01-13 22:47:10,203 | test | DEBUG | MainProcess | MainThread | [bucket_ready_functions:delete_all_buckets:248] Remove bucket default ... 2020-01-13 22:47:10,203 | test | DEBUG | MainProcess | MainThread | [bucket_ready_functions:delete_bucket:171] Deleting existing bucket default on ip:172.23.120.162 port:8091 ssh_username:root 2020-01-13 22:47:10,260 | test | DEBUG | MainProcess | MainThread | [bucket_ready_functions:parse_get_bucket_json:2064] read 1024 vbuckets 2020-01-13 22:47:10,262 | test | DEBUG | MainProcess | MainThread | [bucket_ready_functions:parse_get_bucket_json:2067] Stats: {u'opsPerSec': 0, u'diskFetches': 0, u'dataUsed': 59416032, u'itemCount': 1811, u'quotaPercentUsed': 1.680236384660915, u'storageTotals': {u'hdd': {u'usedByData': 74010680, u'used': 39351700929L, u'quotaTotal': 144901136384L, u'total': 144901136384L, u'free': 75874132132L}, u'ram': {u'usedByData': 94646880, u'used': 13537705984L, u'quotaTotal': 8451522560L, u'total': 16113733632L, u'quotaUsed': 5632950272L, u'quotaUsedPerNode': 1408237568, u'quotaTotalPerNode': 2112880640}}, u'diskUsed': 74010680, u'vbActiveNumNonResident': 0, u'memUsed': 94646880} 2020-01-13 22:47:11,269 | test | DEBUG | MainProcess | MainThread | [bucket_ready_functions:delete_bucket:185] Deleted bucket: default from 172.23.120.162 2020-01-13 22:47:11,270 | test | DEBUG | MainProcess | MainThread | [bucket_ready_functions:wait_for_bucket_deletion:205] Waiting for bucket default deletion to finish 2020-01-13 22:47:11,322 | test | DEBUG | MainProcess | MainThread | [bucket_ready_functions:delete_all_buckets:246] Deleting existing buckets [] on 172.23.120.177 2020-01-13 22:47:11,352 | test | DEBUG | MainProcess | MainThread | [bucket_ready_functions:delete_all_buckets:246] Deleting existing buckets [] on 172.23.120.19 2020-01-13 22:47:11,377 | test | DEBUG | MainProcess | MainThread | [bucket_ready_functions:delete_all_buckets:246] Deleting existing buckets [] on 172.23.120.192 2020-01-13 22:47:11,395 | test | DEBUG | MainProcess | MainThread | [bucket_ready_functions:delete_all_buckets:246] Deleting existing buckets [] on 172.23.121.17 2020-01-13 22:47:11,414 | test | DEBUG | MainProcess | MainThread | [bucket_ready_functions:delete_all_buckets:246] Deleting existing buckets [] on 172.23.121.19 2020-01-13 22:47:11,454 | test | DEBUG | MainProcess | MainThread | [rest_client:is_ns_server_running:44] Checking if ns_server is running with timeout 120secs 2020-01-13 22:47:11,486 | test | DEBUG | MainProcess | MainThread | [cluster_ready_functions:cleanup_cluster:137] Rebalancing all nodes in order to remove nodes 2020-01-13 22:47:11,493 | test | DEBUG | MainProcess | MainThread | [rest_client:remove_nodes:121] Ejecting nodes '[u'ns_1@172.23.120.192', u'ns_1@172.23.120.19', u'ns_1@172.23.120.177']' from cluster 2020-01-13 22:47:11,494 | test | DEBUG | MainProcess | MainThread | [rest_client:rebalance:1327] Rebalance params: {'password': 'password', 'knownNodes': u'ns_1@172.23.120.192,ns_1@172.23.120.19,ns_1@172.23.120.162,ns_1@172.23.120.177', 'ejectedNodes': u'ns_1@172.23.120.192,ns_1@172.23.120.19,ns_1@172.23.120.177', 'user': 'Administrator'} 2020-01-13 22:47:11,506 | test | DEBUG | MainProcess | MainThread | [rest_client:rebalance:1332] Rebalance operation started 2020-01-13 22:47:11,519 | test | DEBUG | MainProcess | MainThread | [rest_client:_rebalance_status_and_progress:1486] Rebalance percentage: 0.00 % 2020-01-13 22:47:36,552 | test | INFO | MainProcess | MainThread | [rest_client:monitorRebalance:1423] Rebalance done. Taken 25.0450000763 seconds to complete 2020-01-13 22:47:36,552 | test | INFO | MainProcess | MainThread | [rest_client:monitorRebalance:1425] Sleep for 5 seconds after rebalance 2020-01-13 22:47:41,640 | test | DEBUG | MainProcess | MainThread | [cluster_ready_functions:cleanup_cluster:183] Removed all the nodes from cluster associated with ip:172.23.120.162 port:8091 ssh_username:root ? [(u'ns_1@172.23.120.192', 8091), (u'ns_1@172.23.120.19', 8091), (u'ns_1@172.23.120.177', 8091)] 2020-01-13 22:47:41,655 | test | DEBUG | MainProcess | MainThread | [cluster_ready_functions:wait_for_ns_servers_or_assert:109] Waiting for ns_server @ 172.23.120.162:8091 2020-01-13 22:47:41,657 | test | DEBUG | MainProcess | MainThread | [rest_client:is_ns_server_running:44] Checking if ns_server is running with timeout 360secs 2020-01-13 22:47:41,671 | test | DEBUG | MainProcess | MainThread | [cluster_ready_functions:wait_for_ns_servers_or_assert:112] ns_server @ 172.23.120.162:8091 is running 2020-01-13 22:47:44,532 | test | DEBUG | MainProcess | MainThread | [cluster_ready_functions:stop_server:360] Couchbase stopped on ip:172.23.120.162 port:8091 ssh_username:root 2020-01-13 22:47:45,163 | test | DEBUG | MainProcess | MainThread | [cluster_ready_functions:start_server:374] Couchbase started on ip:172.23.120.162 port:8091 ssh_username:root 2020-01-13 22:47:47,752 | test | DEBUG | MainProcess | MainThread | [cluster_ready_functions:stop_server:360] Couchbase stopped on ip:172.23.120.177 port:8091 ssh_username:root 2020-01-13 22:47:48,388 | test | DEBUG | MainProcess | MainThread | [cluster_ready_functions:start_server:374] Couchbase started on ip:172.23.120.177 port:8091 ssh_username:root 2020-01-13 22:47:52,203 | test | DEBUG | MainProcess | MainThread | [cluster_ready_functions:stop_server:360] Couchbase stopped on ip:172.23.120.19 port:8091 ssh_username:root 2020-01-13 22:47:53,256 | test | DEBUG | MainProcess | MainThread | [cluster_ready_functions:start_server:374] Couchbase started on ip:172.23.120.19 port:8091 ssh_username:root 2020-01-13 22:47:55,796 | test | DEBUG | MainProcess | MainThread | [cluster_ready_functions:stop_server:360] Couchbase stopped on ip:172.23.120.192 port:8091 ssh_username:root 2020-01-13 22:47:56,434 | test | DEBUG | MainProcess | MainThread | [cluster_ready_functions:start_server:374] Couchbase started on ip:172.23.120.192 port:8091 ssh_username:root 2020-01-13 22:47:58,898 | test | DEBUG | MainProcess | MainThread | [cluster_ready_functions:stop_server:360] Couchbase stopped on ip:172.23.121.17 port:8091 ssh_username:root 2020-01-13 22:47:59,410 | test | DEBUG | MainProcess | MainThread | [cluster_ready_functions:start_server:374] Couchbase started on ip:172.23.121.17 port:8091 ssh_username:root 2020-01-13 22:48:02,227 | test | DEBUG | MainProcess | MainThread | [cluster_ready_functions:stop_server:360] Couchbase stopped on ip:172.23.121.19 port:8091 ssh_username:root 2020-01-13 22:48:02,716 | test | DEBUG | MainProcess | MainThread | [cluster_ready_functions:start_server:374] Couchbase started on ip:172.23.121.19 port:8091 ssh_username:root