guides/gradlew --refresh-dependencies testrunner -P jython=/opt/jython/bin/jython -P 'args=-i /tmp/testexec.6343.ini rerun=False,data_spec_name=initial_load_N1ql,get-cbcollect-info=True,upgrade_version=7.0.0-4226 -t N1qlTransaction.basic_ops.BasicOps.test_with_use_keys,nodes_init=4,num_stmt_txn=5,num_savepoints=2,atrcollection=True,services_init=kv-n1ql-index-fts,GROUP=P0,bucket_spec=single_bucket.def_scope_fifty_collections' Test Input params: {'cluster_name': 'testexec.6343', 'num_stmt_txn': '5', 'ini': '/tmp/testexec.6343.ini', 'get-cbcollect-info': 'True', 'conf_file': 'conf/N1qlTransaction/different_services.conf', 'upgrade_version': '7.0.0-4226', 'atrcollection': 'True', 'spec': 'different_services', 'rerun': 'False', 'num_nodes': 4, 'logs_folder': '/data/workspace/centos-p0-query-vset00-00-N1qltxn_services_P0/logs/testrunner-21-Jan-17_23-06-45/test_3', 'nodes_init': '4', 'num_savepoints': '2', 'GROUP': 'P0', 'services_init': 'kv-n1ql-index-fts', 'bucket_spec': 'single_bucket.def_scope_fifty_collections', 'case_number': 3, 'data_spec_name': 'initial_load_N1ql'} test_with_use_keys (N1qlTransaction.basic_ops.BasicOps) ... 2021-01-17 23:28:05,821 | test | INFO | MainThread | [basetestcase:log_setup_status:554] ========= BaseTestCase setup started for test #3 test_with_use_keys ========= 2021-01-17 23:28:06,351 | test | INFO | MainThread | [basetestcase:log_setup_status:554] ========= BaseTestCase setup started for test #3 test_with_use_keys ========= 2021-01-17 23:28:06,753 | test | INFO | MainThread | [basetestcase:setUp:330] Initializing cluster 2021-01-17 23:28:27,760 | test | WARNING | MainThread | [basetestcase:_initialize_nodes:599] RAM quota was defined less than 100 MB: 2021-01-17 23:28:41,447 | test | INFO | MainThread | [basetestcase:setUp:352] Couchbase Cluster: default, Nodes: 172.23.100.12, 172.23.100.25, 172.23.100.20, 172.23.98.16 initialized 2021-01-17 23:28:41,470 | test | INFO | MainThread | [basetestcase:log_setup_status:554] ========= BaseTestCase setup finished for test #3 test_with_use_keys ========= 2021-01-17 23:28:41,470 | test | WARNING | MainThread | [basetestcase:_BaseTestCase__log:551] Exception during REST log_client_error: type object 'datetime' has no attribute 'datetime' 2021-01-17 23:28:41,470 | test | INFO | MainThread | [basetestcase:log_setup_status:554] ========= CollectionBase setup started for test #3 test_with_use_keys ========= 2021-01-17 23:28:48,019 | test | INFO | pool-4-thread-5 | [table_view:display:72] Rebalance Overview +---------------+-----------+--------------+ | Nodes | Services | Status | +---------------+-----------+--------------+ | 172.23.100.12 | kv | Cluster node | | 172.23.100.25 | ['n1ql'] | <--- IN --- | | 172.23.100.20 | ['index'] | <--- IN --- | | 172.23.98.16 | ['fts'] | <--- IN --- | +---------------+-----------+--------------+ 2021-01-17 23:28:48,039 | test | INFO | pool-4-thread-5 | [task:check:320] Rebalance - status: running, progress: 0 2021-01-17 23:28:58,108 | test | INFO | pool-4-thread-5 | [task:check:320] Rebalance - status: none, progress: 100 2021-01-17 23:28:58,132 | test | INFO | pool-4-thread-5 | [task:check:377] Rebalance completed with progress: 100% in 10.1129999161 sec 2021-01-17 23:28:58,362 | test | INFO | MainThread | [bucket_ready_functions:create_buckets_using_json_data:1775] Creating required buckets from template 2021-01-17 23:28:58,578 | test | INFO | Thread-335 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection j-58-408000 creation 2021-01-17 23:29:00,608 | test | INFO | Thread-335 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection g0EtISpDv9AOyJI-58-408000 creation 2021-01-17 23:29:02,638 | test | INFO | Thread-335 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection d_-58-407000 creation 2021-01-17 23:29:04,665 | test | INFO | Thread-335 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection e%jkb-58-407000 creation 2021-01-17 23:29:06,697 | test | INFO | Thread-335 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection S2jZR7Awj%rOMN%ZGVM-58-408000 creation 2021-01-17 23:29:08,726 | test | INFO | Thread-335 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection evK4bT0zv3QiD-58-407000 creation 2021-01-17 23:29:10,757 | test | INFO | Thread-335 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection pX3UeKrDO-mdj-58-408000 creation 2021-01-17 23:29:12,789 | test | INFO | Thread-335 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection 6IaZPG42CevEkRkG-58-407000 creation 2021-01-17 23:29:14,822 | test | INFO | Thread-335 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection L-58-407000 creation 2021-01-17 23:29:16,878 | test | INFO | Thread-345 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection mj_Usxdf64JUhXQtO4NP-58-414000 creation 2021-01-17 23:29:18,910 | test | INFO | Thread-345 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection miv0gA7QzS14dRjq-58-414000 creation 2021-01-17 23:29:20,940 | test | INFO | Thread-345 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection gEeB%-58-414000 creation 2021-01-17 23:29:22,971 | test | INFO | Thread-345 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection wxXr-58-414000 creation 2021-01-17 23:29:25,000 | test | INFO | Thread-345 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection -gKxRe5JgzzovS_9BQSt-58-415000 creation 2021-01-17 23:29:27,032 | test | INFO | Thread-345 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection qnuho-9-58-415000 creation 2021-01-17 23:29:29,062 | test | INFO | Thread-345 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection xgMIW3fQ3z4hjpyqGdHW-58-415000 creation 2021-01-17 23:29:31,099 | test | INFO | Thread-345 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection aF-58-414000 creation 2021-01-17 23:29:33,131 | test | INFO | Thread-345 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection wm6Kk9q-V0P-58-414000 creation 2021-01-17 23:29:35,161 | test | INFO | Thread-345 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection r1OepYnlpvT2qjTbe-58-415000 creation 2021-01-17 23:29:37,219 | test | INFO | Thread-356 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection NqNl-58-412000 creation 2021-01-17 23:29:39,253 | test | INFO | Thread-356 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection w8j3C-58-411000 creation 2021-01-17 23:29:41,283 | test | INFO | Thread-356 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection 3qLzN11DlSY-58-411000 creation 2021-01-17 23:29:43,315 | test | INFO | Thread-356 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection 3YSGmKeHKmKy-58-411000 creation 2021-01-17 23:29:45,345 | test | INFO | Thread-356 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection FYzz9WTf-58-411000 creation 2021-01-17 23:29:47,377 | test | INFO | Thread-356 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection giziRjGLhm-EYv8Tn-58-410000 creation 2021-01-17 23:29:49,405 | test | INFO | Thread-356 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection g-58-411000 creation 2021-01-17 23:29:51,437 | test | INFO | Thread-356 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection 996_U5WrL9xNFOXQw-58-411000 creation 2021-01-17 23:29:53,471 | test | INFO | Thread-356 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection O4lLi99g1Pbal--58-410000 creation 2021-01-17 23:29:55,503 | test | INFO | Thread-356 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection cn3ytL-sKAqj1vumpO-58-412000 creation 2021-01-17 23:29:57,562 | test | INFO | Thread-367 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection s%2sB9kMevOHRgWF-58-413000 creation 2021-01-17 23:29:59,592 | test | INFO | Thread-367 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection CXo9eEWpbGnHHSZM-58-413000 creation 2021-01-17 23:30:01,627 | test | INFO | Thread-367 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection Wg-58-413000 creation 2021-01-17 23:30:03,661 | test | INFO | Thread-367 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection pHPbGA7-58-413000 creation 2021-01-17 23:30:05,694 | test | INFO | Thread-367 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection Z1Fpm-58-412000 creation 2021-01-17 23:30:07,723 | test | INFO | Thread-367 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection h8jcYcK4B1zqDCfBX9-58-412000 creation 2021-01-17 23:30:09,756 | test | INFO | Thread-367 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection 0mNzm--58-413000 creation 2021-01-17 23:30:11,788 | test | INFO | Thread-367 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection SDbZKFoK-58-412000 creation 2021-01-17 23:30:13,818 | test | INFO | Thread-367 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection s-58-413000 creation 2021-01-17 23:30:15,848 | test | INFO | Thread-367 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection IdM-olI-58-412000 creation 2021-01-17 23:30:17,907 | test | INFO | Thread-378 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection fX6XyGqjCKZYH7V_-58-409000 creation 2021-01-17 23:30:19,937 | test | INFO | Thread-378 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection JhZhvhCs7g-58-409000 creation 2021-01-17 23:30:21,967 | test | INFO | Thread-378 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection l-58-410000 creation 2021-01-17 23:30:24,000 | test | INFO | Thread-378 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection xXD4S%Quwhg_-58-410000 creation 2021-01-17 23:30:26,029 | test | INFO | Thread-378 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection xbBYv2Q7DwR7m-58-409000 creation 2021-01-17 23:30:28,062 | test | INFO | Thread-378 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection ud-58-409000 creation 2021-01-17 23:30:30,095 | test | INFO | Thread-378 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection 98u25hm-NHO-58-410000 creation 2021-01-17 23:30:32,127 | test | INFO | Thread-378 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection SYn2N9EJCrs1UtVb31A-58-409000 creation 2021-01-17 23:30:34,158 | test | INFO | Thread-378 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection GiiqNWGttAy48%eLo79-58-410000 creation 2021-01-17 23:30:36,193 | test | INFO | Thread-378 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection wsOBqEwDHUEvX7DLDv-58-409000 creation 2021-01-17 23:30:38,196 | test | INFO | MainThread | [bucket_ready_functions:wait_for_collection_creation_to_complete:2241] Waiting for all collections to be created 2021-01-17 23:30:38,295 | test | INFO | MainThread | [table_view:display:72] Bucket statistics +------------------------+-----------+----------+------------+-----+-------+------------+----------+-----------+ | Bucket | Type | Replicas | Durability | TTL | Items | RAM Quota | RAM Used | Disk Used | +------------------------+-----------+----------+------------+-----+-------+------------+----------+-----------+ | L5VT2_xKXtFB-58-406000 | couchbase | 2 | none | 0 | 0 | 2929721344 | 38087888 | 44279343 | +------------------------+-----------+----------+------------+-----+-------+------------+----------+-----------+ 2021-01-17 23:30:38,295 | test | INFO | MainThread | [collections_base:collection_setup:143] Creating required SDK clients for client_pool 2021-01-17 23:30:46,565 | test | INFO | MainThread | [common_lib:sleep:22] Sleep 120 seconds. Reason: MB-38497 2021-01-17 23:32:46,566 | test | INFO | MainThread | [bucket_ready_functions:perform_tasks_from_spec:4890] Performing scope/collection specific operations 2021-01-17 23:35:27,891 | test | INFO | pool-4-thread-8 | [table_view:display:72] Ops trend for bucket 'L5VT2_xKXtFB-58-406000' +-----------+-------------------------------------------------------------------------------------------------------------------------------------------------+-----------+ | Min | Trend | Max | +-----------+-------------------------------------------------------------------------------------------------------------------------------------------------+-----------+ | 0.000 | ****................................****...............*******.................................................*******...**************.******X | 15740.000 | | 15619.600 | ..................................................................................................***X | 15619.600 | +-----------+-------------------------------------------------------------------------------------------------------------------------------------------------+-----------+ 2021-01-17 23:35:27,956 | test | INFO | MainThread | [table_view:display:72] Cluster statistics +---------------+----------+-----------------+------------+------------+------------------------+-------------------+-----------------------+ | Node | Services | CPU_utilization | Mem_total | Mem_free | Swap_mem_used | Active / Replica | Version | +---------------+----------+-----------------+------------+------------+------------------------+-------------------+-----------------------+ | 172.23.100.25 | n1ql | 0.980392156863 | 4201684992 | 3565858816 | 0 / 3758092288 | 0 / 0 | 7.0.0-4226-enterprise | | 172.23.100.20 | index | 1.22930255896 | 4201684992 | 3499876352 | 0 / 3758092288 | 0 / 0 | 7.0.0-4226-enterprise | | 172.23.98.16 | fts | 0.652119388011 | 4339109888 | 3715858432 | 0 / 3758092288 | 0 / 0 | 7.0.0-4226-enterprise | | 172.23.100.12 | kv | 68.9834024896 | 4201684992 | 3057606656 | 250085376 / 3758092288 | 903572 / 0 | 7.0.0-4226-enterprise | +---------------+----------+-----------------+------------+------------+------------------------+-------------------+-----------------------+ 2021-01-17 23:35:28,661 | test | INFO | MainThread | [bucket_ready_functions:validate_docs_per_collections_all_buckets:4375] Validating collection stats and item counts 2021-01-17 23:35:31,926 | test | INFO | MainThread | [table_view:display:72] Bucket statistics +------------------------+-----------+----------+------------+-----+--------+------------+-----------+-----------+ | Bucket | Type | Replicas | Durability | TTL | Items | RAM Quota | RAM Used | Disk Used | +------------------------+-----------+----------+------------+-----+--------+------------+-----------+-----------+ | L5VT2_xKXtFB-58-406000 | couchbase | 2 | none | 0 | 903572 | 2929721344 | 359146688 | 558568940 | +------------------------+-----------+----------+------------+-----+--------+------------+-----------+-----------+ 2021-01-17 23:35:31,927 | test | INFO | MainThread | [basetestcase:log_setup_status:554] ========= CollectionBase setup complete for test #3 test_with_use_keys ========= 2021-01-17 23:35:33,000 | test | INFO | MainThread | [tuq_helper:run_cbq_query:147] RUN QUERY CREATE PRIMARY INDEX on default:`L5VT2_xKXtFB-58-406000`.`QuHJ7IU_xuFW-58-410000`.`996_U5WrL9xNFOXQw-58-411000` USING GSI 2021-01-17 23:35:33,000 | test | INFO | MainThread | [tuq_helper:run_cbq_query:163] query_params is {'creds': [{'pass': 'password', 'user': 'Administrator'}, {'pass': u'8b66838a2bb1871d0bac6fb018c0e9b2', 'user': 'local:L5VT2_xKXtFB-58-406000'}], 'scan_consistency': 'REQUEST_PLUS'} 2021-01-17 23:35:35,088 | test | INFO | MainThread | [tuq_helper:run_cbq_query:204] TOTAL ELAPSED TIME: 2.078505916s 2021-01-17 23:35:35,111 | test | INFO | MainThread | [tuq_helper:run_cbq_query:147] RUN QUERY CREATE INDEX `8cs-KcGxNsb-35-89000` ON default:`L5VT2_xKXtFB-58-406000`.`QuHJ7IU_xuFW-58-410000`.`996_U5WrL9xNFOXQw-58-411000`(citizen_of,job_title) USING GSI 2021-01-17 23:35:35,111 | test | INFO | MainThread | [tuq_helper:run_cbq_query:163] query_params is {'creds': [{'pass': 'password', 'user': 'Administrator'}, {'pass': u'8b66838a2bb1871d0bac6fb018c0e9b2', 'user': 'local:L5VT2_xKXtFB-58-406000'}], 'scan_consistency': 'REQUEST_PLUS'} 2021-01-17 23:35:38,142 | test | INFO | MainThread | [tuq_helper:run_cbq_query:204] TOTAL ELAPSED TIME: 3.021664684s 2021-01-17 23:35:38,164 | test | INFO | MainThread | [tuq_helper:run_cbq_query:147] RUN QUERY CREATE INDEX `jvnG1ji4u-xIBjT1c-38-144000` ON default:`L5VT2_xKXtFB-58-406000`.`QuHJ7IU_xuFW-58-410000`.`996_U5WrL9xNFOXQw-58-411000`(citizen_of) USING GSI 2021-01-17 23:35:38,164 | test | INFO | MainThread | [tuq_helper:run_cbq_query:163] query_params is {'creds': [{'pass': 'password', 'user': 'Administrator'}, {'pass': u'8b66838a2bb1871d0bac6fb018c0e9b2', 'user': 'local:L5VT2_xKXtFB-58-406000'}], 'scan_consistency': 'REQUEST_PLUS'} 2021-01-17 23:35:40,917 | test | INFO | MainThread | [tuq_helper:run_cbq_query:204] TOTAL ELAPSED TIME: 2.743622669s 2021-01-17 23:35:40,941 | test | INFO | MainThread | [tuq_helper:run_cbq_query:147] RUN QUERY CREATE INDEX `NOhfJ_-cxVQ7SQKEsWu3-40-918000` ON default:`L5VT2_xKXtFB-58-406000`.`QuHJ7IU_xuFW-58-410000`.`996_U5WrL9xNFOXQw-58-411000`(citizen_of) USING GSI 2021-01-17 23:35:40,941 | test | INFO | MainThread | [tuq_helper:run_cbq_query:163] query_params is {'creds': [{'pass': 'password', 'user': 'Administrator'}, {'pass': u'8b66838a2bb1871d0bac6fb018c0e9b2', 'user': 'local:L5VT2_xKXtFB-58-406000'}], 'scan_consistency': 'REQUEST_PLUS'} 2021-01-17 23:35:43,737 | test | INFO | MainThread | [tuq_helper:run_cbq_query:204] TOTAL ELAPSED TIME: 2.787274001s 2021-01-17 23:35:43,760 | test | INFO | MainThread | [tuq_helper:run_cbq_query:147] RUN QUERY CREATE INDEX `cq9nhQ0qVWLHxKsZeX-43-739000` ON default:`L5VT2_xKXtFB-58-406000`.`QuHJ7IU_xuFW-58-410000`.`996_U5WrL9xNFOXQw-58-411000`(join_day,join_mo) USING GSI 2021-01-17 23:35:43,762 | test | INFO | MainThread | [tuq_helper:run_cbq_query:163] query_params is {'creds': [{'pass': 'password', 'user': 'Administrator'}, {'pass': u'8b66838a2bb1871d0bac6fb018c0e9b2', 'user': 'local:L5VT2_xKXtFB-58-406000'}], 'scan_consistency': 'REQUEST_PLUS'} 2021-01-17 23:35:46,596 | test | INFO | MainThread | [tuq_helper:run_cbq_query:204] TOTAL ELAPSED TIME: 2.826495242s 2021-01-17 23:35:46,621 | test | INFO | MainThread | [tuq_helper:run_cbq_query:147] RUN QUERY CREATE INDEX `PKB_DgMKkI8_AkboOKVs-46-598000` ON default:`L5VT2_xKXtFB-58-406000`.`QuHJ7IU_xuFW-58-410000`.`996_U5WrL9xNFOXQw-58-411000`(Working_country) USING GSI 2021-01-17 23:35:46,621 | test | INFO | MainThread | [tuq_helper:run_cbq_query:163] query_params is {'creds': [{'pass': 'password', 'user': 'Administrator'}, {'pass': u'8b66838a2bb1871d0bac6fb018c0e9b2', 'user': 'local:L5VT2_xKXtFB-58-406000'}], 'scan_consistency': 'REQUEST_PLUS'} 2021-01-17 23:35:49,444 | test | INFO | MainThread | [tuq_helper:run_cbq_query:204] TOTAL ELAPSED TIME: 2.813703639s 2021-01-17 23:35:49,742 | test | INFO | MainThread | [tuq_helper:run_cbq_query:147] RUN QUERY BEGIN WORK 2021-01-17 23:35:49,743 | test | INFO | MainThread | [tuq_helper:run_cbq_query:163] query_params is {'atrcollection': '`L5VT2_xKXtFB-58-406000`.`MlRjeslzKEinxfizSjz-58-413000`.`qnuho-9-58-415000`', 'creds': [{'pass': 'password', 'user': 'Administrator'}, {'pass': u'8b66838a2bb1871d0bac6fb018c0e9b2', 'user': 'local:L5VT2_xKXtFB-58-406000'}], 'scan_consistency': 'REQUEST_PLUS'} 2021-01-17 23:35:49,752 | test | INFO | MainThread | [tuq_helper:run_cbq_query:204] TOTAL ELAPSED TIME: 1.334453ms 2021-01-17 23:35:49,782 | test | INFO | MainThread | [tuq_helper:run_cbq_query:147] RUN QUERY INSERT INTO default:`L5VT2_xKXtFB-58-406000`.`QuHJ7IU_xuFW-58-410000`.`996_U5WrL9xNFOXQw-58-411000` (KEY k1, value t) SELECT DISTINCT t.name AS k1,t FROM default:`L5VT2_xKXtFB-58-406000`.`QuHJ7IU_xuFW-58-410000`.`996_U5WrL9xNFOXQw-58-411000` t WHERE citizen_of='USA' AND job_title='Sales' LIMIT 10 RETURNING * 2021-01-17 23:35:49,782 | test | INFO | MainThread | [tuq_helper:run_cbq_query:163] query_params is {'txid': u'2af0de0b-f216-4d9e-b6ca-d2d639e7a3af', 'creds': [{'pass': 'password', 'user': 'Administrator'}, {'pass': u'8b66838a2bb1871d0bac6fb018c0e9b2', 'user': 'local:L5VT2_xKXtFB-58-406000'}], 'scan_consistency': 'REQUEST_PLUS'} 2021-01-17 23:35:49,805 | test | INFO | MainThread | [tuq_helper:run_cbq_query:204] TOTAL ELAPSED TIME: 13.221961ms 2021-01-17 23:35:49,826 | test | INFO | MainThread | [tuq_helper:run_cbq_query:147] RUN QUERY SELECT * from default:`L5VT2_xKXtFB-58-406000`.`QuHJ7IU_xuFW-58-410000`.`996_U5WrL9xNFOXQw-58-411000` WHERE META().id in ['Treva Juan Jose', 'Fuscienne Bailey', 'Maura Juan Jose', 'Jerica Evans', 'Riona Green', 'Adena Scott', 'Severin Lee', 'Lysha Davis', 'Basha Carter', 'Basha cox'] 2021-01-17 23:35:49,828 | test | INFO | MainThread | [tuq_helper:run_cbq_query:163] query_params is {'txid': u'2af0de0b-f216-4d9e-b6ca-d2d639e7a3af', 'creds': [{'pass': 'password', 'user': 'Administrator'}, {'pass': u'8b66838a2bb1871d0bac6fb018c0e9b2', 'user': 'local:L5VT2_xKXtFB-58-406000'}], 'scan_consistency': 'REQUEST_PLUS'} 2021-01-17 23:35:49,845 | test | INFO | MainThread | [tuq_helper:run_cbq_query:204] TOTAL ELAPSED TIME: 4.861592ms 2021-01-17 23:35:49,868 | test | INFO | MainThread | [tuq_helper:run_cbq_query:147] RUN QUERY INSERT INTO default:`L5VT2_xKXtFB-58-406000`.`QuHJ7IU_xuFW-58-410000`.`996_U5WrL9xNFOXQw-58-411000` (KEY k1, value t) SELECT DISTINCT t.name AS k1,t FROM default:`L5VT2_xKXtFB-58-406000`.`QuHJ7IU_xuFW-58-410000`.`996_U5WrL9xNFOXQw-58-411000` t WHERE citizen_of='INDIA' LIMIT 10 RETURNING * 2021-01-17 23:35:49,868 | test | INFO | MainThread | [tuq_helper:run_cbq_query:163] query_params is {'txid': u'2af0de0b-f216-4d9e-b6ca-d2d639e7a3af', 'creds': [{'pass': 'password', 'user': 'Administrator'}, {'pass': u'8b66838a2bb1871d0bac6fb018c0e9b2', 'user': 'local:L5VT2_xKXtFB-58-406000'}], 'scan_consistency': 'REQUEST_PLUS'} 2021-01-17 23:35:49,881 | test | INFO | MainThread | [tuq_helper:run_cbq_query:204] TOTAL ELAPSED TIME: 3.975581ms 2021-01-17 23:35:49,904 | test | INFO | MainThread | [tuq_helper:run_cbq_query:147] RUN QUERY SELECT * from default:`L5VT2_xKXtFB-58-406000`.`QuHJ7IU_xuFW-58-410000`.`996_U5WrL9xNFOXQw-58-411000` WHERE META().id in [] 2021-01-17 23:35:49,904 | test | INFO | MainThread | [tuq_helper:run_cbq_query:163] query_params is {'txid': u'2af0de0b-f216-4d9e-b6ca-d2d639e7a3af', 'creds': [{'pass': 'password', 'user': 'Administrator'}, {'pass': u'8b66838a2bb1871d0bac6fb018c0e9b2', 'user': 'local:L5VT2_xKXtFB-58-406000'}], 'scan_consistency': 'REQUEST_PLUS'} 2021-01-17 23:35:49,914 | test | INFO | MainThread | [tuq_helper:run_cbq_query:204] TOTAL ELAPSED TIME: 2.976966ms 2021-01-17 23:35:49,936 | test | INFO | MainThread | [tuq_helper:run_cbq_query:147] RUN QUERY SAVEPOINT a0 2021-01-17 23:35:49,936 | test | INFO | MainThread | [tuq_helper:run_cbq_query:163] query_params is {'txid': u'2af0de0b-f216-4d9e-b6ca-d2d639e7a3af', 'creds': [{'pass': 'password', 'user': 'Administrator'}, {'pass': u'8b66838a2bb1871d0bac6fb018c0e9b2', 'user': 'local:L5VT2_xKXtFB-58-406000'}], 'scan_consistency': 'REQUEST_PLUS'} 2021-01-17 23:35:49,944 | test | INFO | MainThread | [tuq_helper:run_cbq_query:204] TOTAL ELAPSED TIME: 414.688µs 2021-01-17 23:35:49,974 | test | INFO | MainThread | [tuq_helper:run_cbq_query:147] RUN QUERY DELETE FROM default:`L5VT2_xKXtFB-58-406000`.`QuHJ7IU_xuFW-58-410000`.`996_U5WrL9xNFOXQw-58-411000` USE KEYS ['test_collections-5352', 'test_collections-271', 'test_collections-16157', 'test_collections-17772', 'test_collections-2571', 'test_collections-14694', 'test_collections-16659', 'test_collections-2217', 'test_collections-8716', 'test_collections-14310', 'test_collections-4039', 'test_collections-19675', 'test_collections-4487', 'test_collections-5556', 'test_collections-14982', 'test_collections-14312', 'test_collections-8442', 'test_collections-15463', 'test_collections-18506', 'test_collections-9445'] RETURNING meta().id 2021-01-17 23:35:49,976 | test | INFO | MainThread | [tuq_helper:run_cbq_query:163] query_params is {'txid': u'2af0de0b-f216-4d9e-b6ca-d2d639e7a3af', 'creds': [{'pass': 'password', 'user': 'Administrator'}, {'pass': u'8b66838a2bb1871d0bac6fb018c0e9b2', 'user': 'local:L5VT2_xKXtFB-58-406000'}], 'scan_consistency': 'REQUEST_PLUS'} 2021-01-17 23:35:49,989 | test | INFO | MainThread | [tuq_helper:run_cbq_query:204] TOTAL ELAPSED TIME: 4.764401ms 2021-01-17 23:35:50,009 | test | INFO | MainThread | [tuq_helper:run_cbq_query:147] RUN QUERY SELECT META().id,* from default:`L5VT2_xKXtFB-58-406000`.`QuHJ7IU_xuFW-58-410000`.`996_U5WrL9xNFOXQw-58-411000` WHERE META().id in ['test_collections-5352', 'test_collections-271', 'test_collections-16157', 'test_collections-17772', 'test_collections-2571', 'test_collections-14694', 'test_collections-16659', 'test_collections-2217', 'test_collections-8716', 'test_collections-14310', 'test_collections-4039', 'test_collections-19675', 'test_collections-4487', 'test_collections-5556', 'test_collections-14982', 'test_collections-14312', 'test_collections-8442', 'test_collections-15463', 'test_collections-18506', 'test_collections-9445'] 2021-01-17 23:35:50,010 | test | INFO | MainThread | [tuq_helper:run_cbq_query:163] query_params is {'txid': u'2af0de0b-f216-4d9e-b6ca-d2d639e7a3af', 'creds': [{'pass': 'password', 'user': 'Administrator'}, {'pass': u'8b66838a2bb1871d0bac6fb018c0e9b2', 'user': 'local:L5VT2_xKXtFB-58-406000'}], 'scan_consistency': 'REQUEST_PLUS'} 2021-01-17 23:35:50,025 | test | INFO | MainThread | [tuq_helper:run_cbq_query:204] TOTAL ELAPSED TIME: 6.302309ms 2021-01-17 23:35:50,026 | test | INFO | MainThread | [N1qlBase:validate_delete_results:153] delete result is [] 2021-01-17 23:35:50,048 | test | INFO | MainThread | [tuq_helper:run_cbq_query:147] RUN QUERY SAVEPOINT a1 2021-01-17 23:35:50,049 | test | INFO | MainThread | [tuq_helper:run_cbq_query:163] query_params is {'txid': u'2af0de0b-f216-4d9e-b6ca-d2d639e7a3af', 'creds': [{'pass': 'password', 'user': 'Administrator'}, {'pass': u'8b66838a2bb1871d0bac6fb018c0e9b2', 'user': 'local:L5VT2_xKXtFB-58-406000'}], 'scan_consistency': 'REQUEST_PLUS'} 2021-01-17 23:35:50,056 | test | INFO | MainThread | [tuq_helper:run_cbq_query:204] TOTAL ELAPSED TIME: 416.321µs 2021-01-17 23:35:50,078 | test | INFO | MainThread | [tuq_helper:run_cbq_query:147] RUN QUERY DELETE FROM default:`L5VT2_xKXtFB-58-406000`.`QuHJ7IU_xuFW-58-410000`.`996_U5WrL9xNFOXQw-58-411000` USE KEYS ['test_collections-15160', 'test_collections-7870', 'test_collections-14133', 'test_collections-6884', 'test_collections-14782', 'test_collections-4312', 'test_collections-13839', 'test_collections-1479', 'test_collections-12507', 'test_collections-11506', 'test_collections-3391', 'test_collections-7096', 'test_collections-10863', 'test_collections-12369', 'test_collections-5736', 'test_collections-585', 'test_collections-12424', 'test_collections-8159', 'test_collections-16796', 'test_collections-6246'] RETURNING meta().id 2021-01-17 23:35:50,079 | test | INFO | MainThread | [tuq_helper:run_cbq_query:163] query_params is {'txid': u'2af0de0b-f216-4d9e-b6ca-d2d639e7a3af', 'creds': [{'pass': 'password', 'user': 'Administrator'}, {'pass': u'8b66838a2bb1871d0bac6fb018c0e9b2', 'user': 'local:L5VT2_xKXtFB-58-406000'}], 'scan_consistency': 'REQUEST_PLUS'} 2021-01-17 23:35:50,095 | test | INFO | MainThread | [tuq_helper:run_cbq_query:204] TOTAL ELAPSED TIME: 6.628394ms 2021-01-17 23:35:50,115 | test | INFO | MainThread | [tuq_helper:run_cbq_query:147] RUN QUERY SELECT META().id,* from default:`L5VT2_xKXtFB-58-406000`.`QuHJ7IU_xuFW-58-410000`.`996_U5WrL9xNFOXQw-58-411000` WHERE META().id in ['test_collections-15160', 'test_collections-7870', 'test_collections-14133', 'test_collections-6884', 'test_collections-14782', 'test_collections-4312', 'test_collections-13839', 'test_collections-1479', 'test_collections-12507', 'test_collections-11506', 'test_collections-3391', 'test_collections-7096', 'test_collections-10863', 'test_collections-12369', 'test_collections-5736', 'test_collections-585', 'test_collections-12424', 'test_collections-8159', 'test_collections-16796', 'test_collections-6246'] 2021-01-17 23:35:50,115 | test | INFO | MainThread | [tuq_helper:run_cbq_query:163] query_params is {'txid': u'2af0de0b-f216-4d9e-b6ca-d2d639e7a3af', 'creds': [{'pass': 'password', 'user': 'Administrator'}, {'pass': u'8b66838a2bb1871d0bac6fb018c0e9b2', 'user': 'local:L5VT2_xKXtFB-58-406000'}], 'scan_consistency': 'REQUEST_PLUS'} 2021-01-17 23:35:50,130 | test | INFO | MainThread | [tuq_helper:run_cbq_query:204] TOTAL ELAPSED TIME: 6.228329ms 2021-01-17 23:35:50,131 | test | INFO | MainThread | [N1qlBase:validate_delete_results:153] delete result is [] 2021-01-17 23:35:50,151 | test | INFO | MainThread | [tuq_helper:run_cbq_query:147] RUN QUERY DELETE FROM default:`L5VT2_xKXtFB-58-406000`.`QuHJ7IU_xuFW-58-410000`.`996_U5WrL9xNFOXQw-58-411000` USE KEYS ['test_collections-6134', 'test_collections-5589', 'test_collections-12295', 'test_collections-4905', 'test_collections-16528', 'test_collections-4655', 'test_collections-13202', 'test_collections-18888', 'test_collections-1855', 'test_collections-456', 'test_collections-6904', 'test_collections-17042', 'test_collections-4484', 'test_collections-5415', 'test_collections-11397', 'test_collections-2389', 'test_collections-4966', 'test_collections-7073', 'test_collections-613', 'test_collections-9093'] RETURNING meta().id 2021-01-17 23:35:50,151 | test | INFO | MainThread | [tuq_helper:run_cbq_query:163] query_params is {'txid': u'2af0de0b-f216-4d9e-b6ca-d2d639e7a3af', 'creds': [{'pass': 'password', 'user': 'Administrator'}, {'pass': u'8b66838a2bb1871d0bac6fb018c0e9b2', 'user': 'local:L5VT2_xKXtFB-58-406000'}], 'scan_consistency': 'REQUEST_PLUS'} 2021-01-17 23:35:50,165 | test | INFO | MainThread | [tuq_helper:run_cbq_query:204] TOTAL ELAPSED TIME: 4.180005ms 2021-01-17 23:35:50,187 | test | INFO | MainThread | [tuq_helper:run_cbq_query:147] RUN QUERY SELECT META().id,* from default:`L5VT2_xKXtFB-58-406000`.`QuHJ7IU_xuFW-58-410000`.`996_U5WrL9xNFOXQw-58-411000` WHERE META().id in ['test_collections-6134', 'test_collections-5589', 'test_collections-12295', 'test_collections-4905', 'test_collections-16528', 'test_collections-4655', 'test_collections-13202', 'test_collections-18888', 'test_collections-1855', 'test_collections-456', 'test_collections-6904', 'test_collections-17042', 'test_collections-4484', 'test_collections-5415', 'test_collections-11397', 'test_collections-2389', 'test_collections-4966', 'test_collections-7073', 'test_collections-613', 'test_collections-9093'] 2021-01-17 23:35:50,187 | test | INFO | MainThread | [tuq_helper:run_cbq_query:163] query_params is {'txid': u'2af0de0b-f216-4d9e-b6ca-d2d639e7a3af', 'creds': [{'pass': 'password', 'user': 'Administrator'}, {'pass': u'8b66838a2bb1871d0bac6fb018c0e9b2', 'user': 'local:L5VT2_xKXtFB-58-406000'}], 'scan_consistency': 'REQUEST_PLUS'} 2021-01-17 23:35:50,200 | test | INFO | MainThread | [tuq_helper:run_cbq_query:204] TOTAL ELAPSED TIME: 6.194615ms 2021-01-17 23:35:50,201 | test | INFO | MainThread | [N1qlBase:validate_delete_results:153] delete result is [] 2021-01-17 23:35:50,223 | test | INFO | MainThread | [tuq_helper:run_cbq_query:147] RUN QUERY SELECT * FROM system:transactions 2021-01-17 23:35:50,223 | test | INFO | MainThread | [tuq_helper:run_cbq_query:163] query_params is {'creds': [{'pass': 'password', 'user': 'Administrator'}, {'pass': u'8b66838a2bb1871d0bac6fb018c0e9b2', 'user': 'local:L5VT2_xKXtFB-58-406000'}], 'scan_consistency': 'REQUEST_PLUS'} 2021-01-17 23:35:50,233 | test | INFO | MainThread | [tuq_helper:run_cbq_query:204] TOTAL ELAPSED TIME: 2.5497ms 2021-01-17 23:35:50,233 | test | INFO | MainThread | [tuq_helper:end_txn:100] commit work for txn {'txid': u'2af0de0b-f216-4d9e-b6ca-d2d639e7a3af', 'scan_consistency': 'REQUEST_PLUS'} 2021-01-17 23:35:50,255 | test | INFO | MainThread | [tuq_helper:run_cbq_query:147] RUN QUERY COMMIT WORK 2021-01-17 23:35:50,255 | test | INFO | MainThread | [tuq_helper:run_cbq_query:163] query_params is {'txid': u'2af0de0b-f216-4d9e-b6ca-d2d639e7a3af', 'creds': [{'pass': 'password', 'user': 'Administrator'}, {'pass': u'8b66838a2bb1871d0bac6fb018c0e9b2', 'user': 'local:L5VT2_xKXtFB-58-406000'}], 'scan_consistency': 'REQUEST_PLUS'} 2021-01-17 23:35:50,265 | test | INFO | MainThread | [tuq_helper:run_cbq_query:204] TOTAL ELAPSED TIME: 2.049293ms 2021-01-17 23:35:50,266 | test | INFO | MainThread | [N1qlBase:full_execute_query:366] txn failed with unexpected errors {u'signature': u'json', u'requestID': u'bfbbee62-b264-43c1-803c-efd4f7259ed6', u'metrics': {u'resultCount': 0, u'executionTime': u'1.978552ms', u'serviceLoad': 1, u'resultSize': 0, u'errorCount': 1, u'elapsedTime': u'2.049293ms'}, u'results': [], u'errors': [{u'msg': u'Commit Transaction statement error', u'code': 17007, u'cause': {u'rollback': False, u'cause': u'durability impossible | {"status_code":161,"document_key":"_txn:atr-275-#128e","bucket":"L5VT2_xKXtFB-58-406000","scope":"MlRjeslzKEinxfizSjz-58-413000","collection":"qnuho-9-58-415000","collection_id":22,"error_name":"DurabilityImpossible","error_description":"Durability requirements are impossible to achieve","opaque":8,"last_dispatched_to":"172.23.100.12:11210","last_dispatched_from":"172.23.100.25:43898","last_connection_id":"4956d10c48164c0b/edc44fe41a8447d2"}', u'raise': u'failed', u'class': u'other', u'retry': False}}], u'status': u'errors'} 2021-01-17 23:35:50,266 | test | INFO | MainThread | [N1qlBase:execute_query_and_validate_results:654] queries ran are {u'2af0de0b-f216-4d9e-b6ca-d2d639e7a3af': ["INSERT INTO default:`L5VT2_xKXtFB-58-406000`.`QuHJ7IU_xuFW-58-410000`.`996_U5WrL9xNFOXQw-58-411000` (KEY k1, value t) SELECT DISTINCT t.name AS k1,t FROM default:`L5VT2_xKXtFB-58-406000`.`QuHJ7IU_xuFW-58-410000`.`996_U5WrL9xNFOXQw-58-411000` t WHERE citizen_of='USA' AND job_title='Sales' LIMIT 10 RETURNING *", "INSERT INTO default:`L5VT2_xKXtFB-58-406000`.`QuHJ7IU_xuFW-58-410000`.`996_U5WrL9xNFOXQw-58-411000` (KEY k1, value t) SELECT DISTINCT t.name AS k1,t FROM default:`L5VT2_xKXtFB-58-406000`.`QuHJ7IU_xuFW-58-410000`.`996_U5WrL9xNFOXQw-58-411000` t WHERE citizen_of='INDIA' LIMIT 10 RETURNING *", 'SAVEPOINT a0', "DELETE FROM default:`L5VT2_xKXtFB-58-406000`.`QuHJ7IU_xuFW-58-410000`.`996_U5WrL9xNFOXQw-58-411000` USE KEYS ['test_collections-5352', 'test_collections-271', 'test_collections-16157', 'test_collections-17772', 'test_collections-2571', 'test_collections-14694', 'test_collections-16659', 'test_collections-2217', 'test_collections-8716', 'test_collections-14310', 'test_collections-4039', 'test_collections-19675', 'test_collections-4487', 'test_collections-5556', 'test_collections-14982', 'test_collections-14312', 'test_collections-8442', 'test_collections-15463', 'test_collections-18506', 'test_collections-9445'] RETURNING meta().id", 'SAVEPOINT a1', "DELETE FROM default:`L5VT2_xKXtFB-58-406000`.`QuHJ7IU_xuFW-58-410000`.`996_U5WrL9xNFOXQw-58-411000` USE KEYS ['test_collections-15160', 'test_collections-7870', 'test_collections-14133', 'test_collections-6884', 'test_collections-14782', 'test_collections-4312', 'test_collections-13839', 'test_collections-1479', 'test_collections-12507', 'test_collections-11506', 'test_collections-3391', 'test_collections-7096', 'test_collections-10863', 'test_collections-12369', 'test_collections-5736', 'test_collections-585', 'test_collections-12424', 'test_collections-8159', 'test_collections-16796', 'test_collections-6246'] RETURNING meta().id", "DELETE FROM default:`L5VT2_xKXtFB-58-406000`.`QuHJ7IU_xuFW-58-410000`.`996_U5WrL9xNFOXQw-58-411000` USE KEYS ['test_collections-6134', 'test_collections-5589', 'test_collections-12295', 'test_collections-4905', 'test_collections-16528', 'test_collections-4655', 'test_collections-13202', 'test_collections-18888', 'test_collections-1855', 'test_collections-456', 'test_collections-6904', 'test_collections-17042', 'test_collections-4484', 'test_collections-5415', 'test_collections-11397', 'test_collections-2389', 'test_collections-4966', 'test_collections-7073', 'test_collections-613', 'test_collections-9093'] RETURNING meta().id", 'SELECT * FROM system:transactions']} FAIL 2021-01-17 23:35:51,176 | test | INFO | MainThread | [tuq_helper:run_cbq_query:147] RUN QUERY DROP INDEX default:`L5VT2_xKXtFB-58-406000`.`QuHJ7IU_xuFW-58-410000`.`996_U5WrL9xNFOXQw-58-411000`.`8cs-KcGxNsb-35-89000` USING GSI 2021-01-17 23:35:51,176 | test | INFO | MainThread | [tuq_helper:run_cbq_query:163] query_params is {'creds': [{'pass': 'password', 'user': 'Administrator'}, {'pass': u'8b66838a2bb1871d0bac6fb018c0e9b2', 'user': 'local:L5VT2_xKXtFB-58-406000'}], 'scan_consistency': 'REQUEST_PLUS'} 2021-01-17 23:35:51,369 | test | INFO | MainThread | [tuq_helper:run_cbq_query:204] TOTAL ELAPSED TIME: 184.759932ms 2021-01-17 23:35:51,391 | test | INFO | MainThread | [tuq_helper:run_cbq_query:147] RUN QUERY DROP INDEX default:`L5VT2_xKXtFB-58-406000`.`QuHJ7IU_xuFW-58-410000`.`996_U5WrL9xNFOXQw-58-411000`.`jvnG1ji4u-xIBjT1c-38-144000` USING GSI 2021-01-17 23:35:51,391 | test | INFO | MainThread | [tuq_helper:run_cbq_query:163] query_params is {'creds': [{'pass': 'password', 'user': 'Administrator'}, {'pass': u'8b66838a2bb1871d0bac6fb018c0e9b2', 'user': 'local:L5VT2_xKXtFB-58-406000'}], 'scan_consistency': 'REQUEST_PLUS'} 2021-01-17 23:35:51,421 | test | INFO | MainThread | [tuq_helper:run_cbq_query:204] TOTAL ELAPSED TIME: 20.901539ms 2021-01-17 23:35:51,443 | test | INFO | MainThread | [tuq_helper:run_cbq_query:147] RUN QUERY DROP INDEX default:`L5VT2_xKXtFB-58-406000`.`QuHJ7IU_xuFW-58-410000`.`996_U5WrL9xNFOXQw-58-411000`.`NOhfJ_-cxVQ7SQKEsWu3-40-918000` USING GSI 2021-01-17 23:35:51,443 | test | INFO | MainThread | [tuq_helper:run_cbq_query:163] query_params is {'creds': [{'pass': 'password', 'user': 'Administrator'}, {'pass': u'8b66838a2bb1871d0bac6fb018c0e9b2', 'user': 'local:L5VT2_xKXtFB-58-406000'}], 'scan_consistency': 'REQUEST_PLUS'} 2021-01-17 23:35:51,473 | test | INFO | MainThread | [tuq_helper:run_cbq_query:204] TOTAL ELAPSED TIME: 21.93115ms 2021-01-17 23:35:51,493 | test | INFO | MainThread | [tuq_helper:run_cbq_query:147] RUN QUERY DROP INDEX default:`L5VT2_xKXtFB-58-406000`.`QuHJ7IU_xuFW-58-410000`.`996_U5WrL9xNFOXQw-58-411000`.`cq9nhQ0qVWLHxKsZeX-43-739000` USING GSI 2021-01-17 23:35:51,494 | test | INFO | MainThread | [tuq_helper:run_cbq_query:163] query_params is {'creds': [{'pass': 'password', 'user': 'Administrator'}, {'pass': u'8b66838a2bb1871d0bac6fb018c0e9b2', 'user': 'local:L5VT2_xKXtFB-58-406000'}], 'scan_consistency': 'REQUEST_PLUS'} 2021-01-17 23:35:51,529 | test | INFO | MainThread | [tuq_helper:run_cbq_query:204] TOTAL ELAPSED TIME: 26.780063ms 2021-01-17 23:35:51,549 | test | INFO | MainThread | [tuq_helper:run_cbq_query:147] RUN QUERY DROP INDEX default:`L5VT2_xKXtFB-58-406000`.`QuHJ7IU_xuFW-58-410000`.`996_U5WrL9xNFOXQw-58-411000`.`PKB_DgMKkI8_AkboOKVs-46-598000` USING GSI 2021-01-17 23:35:51,549 | test | INFO | MainThread | [tuq_helper:run_cbq_query:163] query_params is {'creds': [{'pass': 'password', 'user': 'Administrator'}, {'pass': u'8b66838a2bb1871d0bac6fb018c0e9b2', 'user': 'local:L5VT2_xKXtFB-58-406000'}], 'scan_consistency': 'REQUEST_PLUS'} 2021-01-17 23:35:51,585 | test | INFO | MainThread | [tuq_helper:run_cbq_query:204] TOTAL ELAPSED TIME: 27.127598ms 2021-01-17 23:35:52,155 | test | INFO | MainThread | [collections_base:tearDown:90] Bucket: L5VT2_xKXtFB-58-406000, Active Resident ratio(DGM): 100% 2021-01-17 23:35:52,157 | test | INFO | MainThread | [collections_base:tearDown:93] Bucket: L5VT2_xKXtFB-58-406000, Replica Resident ratio(DGM): 100% 2021-01-17 23:35:56,262 | test | INFO | MainThread | [basetestcase:check_coredump_exist:708] 172.23.100.12: Looking for crash / dump files 2021-01-17 23:35:56,341 | test | INFO | MainThread | [basetestcase:check_coredump_exist:763] Looking for CRITICAL|WARN messages in /opt/couchbase/var/lib/couchbase/logs/memcached.log.000000.txt 2021-01-17 23:35:56,434 | test | INFO | MainThread | [basetestcase:check_coredump_exist:763] Looking for CRITICAL|WARN messages in /opt/couchbase/var/lib/couchbase/logs/memcached.log.000001.txt 2021-01-17 23:35:56,520 | test | INFO | MainThread | [basetestcase:check_coredump_exist:763] Looking for CRITICAL|WARN messages in /opt/couchbase/var/lib/couchbase/logs/memcached.log.000002.txt 2021-01-17 23:35:56,607 | test | INFO | MainThread | [basetestcase:check_coredump_exist:763] Looking for CRITICAL|WARN messages in /opt/couchbase/var/lib/couchbase/logs/memcached.log.000003.txt 2021-01-17 23:35:56,696 | test | INFO | MainThread | [basetestcase:check_coredump_exist:763] Looking for CRITICAL|WARN messages in /opt/couchbase/var/lib/couchbase/logs/memcached.log.000004.txt 2021-01-17 23:35:56,786 | test | INFO | MainThread | [basetestcase:check_coredump_exist:763] Looking for CRITICAL|WARN messages in /opt/couchbase/var/lib/couchbase/logs/memcached.log.000005.txt 2021-01-17 23:35:56,875 | test | INFO | MainThread | [basetestcase:check_coredump_exist:763] Looking for CRITICAL|WARN messages in /opt/couchbase/var/lib/couchbase/logs/memcached.log.000006.txt 2021-01-17 23:35:56,963 | test | INFO | MainThread | [basetestcase:check_coredump_exist:763] Looking for CRITICAL|WARN messages in /opt/couchbase/var/lib/couchbase/logs/memcached.log.000007.txt 2021-01-17 23:35:57,052 | test | INFO | MainThread | [basetestcase:check_coredump_exist:763] Looking for CRITICAL|WARN messages in /opt/couchbase/var/lib/couchbase/logs/memcached.log.000008.txt 2021-01-17 23:35:57,145 | test | INFO | MainThread | [basetestcase:check_coredump_exist:763] Looking for CRITICAL|WARN messages in /opt/couchbase/var/lib/couchbase/logs/memcached.log.000009.txt 2021-01-17 23:35:57,246 | test | INFO | MainThread | [basetestcase:check_coredump_exist:763] Looking for CRITICAL|WARN messages in /opt/couchbase/var/lib/couchbase/logs/memcached.log.000010.txt 2021-01-17 23:35:57,335 | test | INFO | MainThread | [basetestcase:check_coredump_exist:763] Looking for CRITICAL|WARN messages in /opt/couchbase/var/lib/couchbase/logs/memcached.log.000011.txt 2021-01-17 23:35:57,423 | test | INFO | MainThread | [basetestcase:check_coredump_exist:763] Looking for CRITICAL|WARN messages in /opt/couchbase/var/lib/couchbase/logs/memcached.log.000012.txt 2021-01-17 23:35:57,513 | test | INFO | MainThread | [basetestcase:check_coredump_exist:763] Looking for CRITICAL|WARN messages in /opt/couchbase/var/lib/couchbase/logs/memcached.log.000013.txt 2021-01-17 23:35:57,598 | test | WARNING | MainThread | [basetestcase:check_coredump_exist:752] 172.23.100.12: No 'sanitizers.log.*' files found 2021-01-17 23:35:58,061 | test | INFO | MainThread | [basetestcase:check_coredump_exist:708] 172.23.100.25: Looking for crash / dump files 2021-01-17 23:35:58,131 | test | INFO | MainThread | [basetestcase:check_coredump_exist:763] Looking for CRITICAL|WARN messages in /opt/couchbase/var/lib/couchbase/logs/memcached.log.000000.txt 2021-01-17 23:35:58,207 | test | WARNING | MainThread | [basetestcase:check_coredump_exist:752] 172.23.100.25: No 'sanitizers.log.*' files found 2021-01-17 23:35:58,674 | test | INFO | MainThread | [basetestcase:check_coredump_exist:708] 172.23.100.20: Looking for crash / dump files 2021-01-17 23:35:58,747 | test | INFO | MainThread | [basetestcase:check_coredump_exist:763] Looking for CRITICAL|WARN messages in /opt/couchbase/var/lib/couchbase/logs/memcached.log.000000.txt 2021-01-17 23:35:58,822 | test | WARNING | MainThread | [basetestcase:check_coredump_exist:752] 172.23.100.20: No 'sanitizers.log.*' files found 2021-01-17 23:35:59,243 | test | INFO | MainThread | [basetestcase:check_coredump_exist:708] 172.23.98.16: Looking for crash / dump files 2021-01-17 23:35:59,311 | test | INFO | MainThread | [basetestcase:check_coredump_exist:763] Looking for CRITICAL|WARN messages in /opt/couchbase/var/lib/couchbase/logs/memcached.log.000000.txt 2021-01-17 23:35:59,368 | test | WARNING | MainThread | [basetestcase:check_coredump_exist:752] 172.23.98.16: No 'sanitizers.log.*' files found 2021-01-17 23:35:59,460 | test | INFO | MainThread | [cluster_ready_functions:trigger_cb_collect_on_cluster:1071] Running cbcollect on node ns_1@172.23.100.12,ns_1@172.23.100.20,ns_1@172.23.100.25,ns_1@172.23.98.16 2021-01-17 23:36:09,497 | test | INFO | MainThread | [cluster_ready_functions:trigger_cb_collect_on_cluster:1074] ns_1@172.23.100.12,ns_1@172.23.100.20,ns_1@172.23.100.25,ns_1@172.23.98.16 - cbcollect status: True 2021-01-17 23:36:09,499 | test | INFO | MainThread | [cluster_ready_functions:wait_for_cb_collect_to_complete:1079] Polling active_tasks to check cbcollect status 2021-01-17 23:37:59,894 | test | INFO | MainThread | [cluster_ready_functions:copy_cb_collect_logs:1102] ns_1@172.23.100.12: Copying cbcollect ZIP file to Client 2021-01-17 23:38:00,661 | test | ERROR | MainThread | [cluster_ready_functions:copy_cb_collect_logs:1123] 172.23.100.12 node cb collect zip copied on client : True 2021-01-17 23:38:00,661 | test | INFO | MainThread | [cluster_ready_functions:copy_cb_collect_logs:1102] ns_1@172.23.100.20: Copying cbcollect ZIP file to Client 2021-01-17 23:38:01,276 | test | ERROR | MainThread | [cluster_ready_functions:copy_cb_collect_logs:1123] 172.23.100.20 node cb collect zip copied on client : True 2021-01-17 23:38:01,278 | test | INFO | MainThread | [cluster_ready_functions:copy_cb_collect_logs:1102] ns_1@172.23.100.25: Copying cbcollect ZIP file to Client 2021-01-17 23:38:01,834 | test | ERROR | MainThread | [cluster_ready_functions:copy_cb_collect_logs:1123] 172.23.100.25 node cb collect zip copied on client : True 2021-01-17 23:38:01,835 | test | INFO | MainThread | [cluster_ready_functions:copy_cb_collect_logs:1102] ns_1@172.23.98.16: Copying cbcollect ZIP file to Client 2021-01-17 23:38:02,329 | test | ERROR | MainThread | [cluster_ready_functions:copy_cb_collect_logs:1123] 172.23.98.16 node cb collect zip copied on client : True 2021-01-17 23:38:02,329 | test | CRITICAL | MainThread | [basetestcase:tearDownEverything:499] Skipping get_trace !! 2021-01-17 23:38:33,471 | test | INFO | MainThread | [rest_client:monitorRebalance:1438] Rebalance done. Taken 29.3089997768 seconds to complete 2021-01-17 23:38:33,486 | test | INFO | MainThread | [common_lib:sleep:22] Sleep 5 seconds. Reason: Wait after rebalance complete ====================================================================== FAIL: test_with_use_keys (N1qlTransaction.basic_ops.BasicOps) ---------------------------------------------------------------------- Traceback (most recent call last): File "pytests/N1qlTransaction/basic_ops.py", line 67, in test_with_use_keys self.execute_query_and_validate_results(stmts, File "./pytests/N1qlTransaction/N1qlBase.py", line 662, in execute_query_and_validate_results self.fail(collection_savepoint) AssertionError: txn failed with unexpected errors {u'signature': u'json', u'requestID': u'bfbbee62-b264-43c1-803c-efd4f7259ed6', u'metrics': {u'resultCount': 0, u'executionTime': u'1.978552ms', u'serviceLoad': 1, u'resultSize': 0, u'errorCount': 1, u'elapsedTime': u'2.049293ms'}, u'results': [], u'errors': [{u'msg': u'Commit Transaction statement error', u'code': 17007, u'cause': {u'rollback': False, u'cause': u'durability impossible | {"status_code":161,"document_key":"_txn:atr-275-#128e","bucket":"L5VT2_xKXtFB-58-406000","scope":"MlRjeslzKEinxfizSjz-58-413000","collection":"qnuho-9-58-415000","collection_id":22,"error_name":"DurabilityImpossible","error_description":"Durability requirements are impossible to achieve","opaque":8,"last_dispatched_to":"172.23.100.12:11210","last_dispatched_from":"172.23.100.25:43898","last_connection_id":"4956d10c48164c0b/edc44fe41a8447d2"}', u'raise': u'failed', u'class': u'other', u'retry': False}}], u'status': u'errors'} ---------------------------------------------------------------------- Ran 1 test in 658.902s FAILED (failures=1) During the test, Remote Connections: 39, Disconnections: 39 SDK Connections: 30, Disconnections: 30 summary so far suite N1qlTransaction.basic_ops.BasicOps , pass 0 , fail 3