./testrunner -i /tmp/atomicity.ini rerun=False,data_spec_name=initial_load_N1ql,get-cbcollect-info=True -t N1qlTransaction.basic_ops.BasicOps.test_concurrent_txn,nodes_init=2,num_stmt_txn=10,num_collection=1,num_savepoints=2,override_savepoint=1,num_txn=3,num_commit=3,bucket_spec=single_bucket.def_scope_ten_collections Test Input params: {'cluster_name': 'atomicity', 'num_stmt_txn': '10', 'ini': '/tmp/atomicity.ini', 'get-cbcollect-info': 'True', 'conf_file': 'conf/N1qlTransaction/concurrency.conf', 'spec': 'concurrency', 'rerun': 'False', 'num_nodes': 2, 'logs_folder': '/data/workspace/temp_atomicity_phase1/logs/testrunner-20-Oct-02_05-19-23/test_3', 'num_txn': '3', 'nodes_init': '2', 'num_collection': '1', 'num_savepoints': '2', 'num_commit': '3', 'override_savepoint': '1', 'bucket_spec': 'single_bucket.def_scope_ten_collections', 'case_number': 3, 'data_spec_name': 'initial_load_N1ql'} test_concurrent_txn (N1qlTransaction.basic_ops.BasicOps) ... 2020-10-02 05:27:33,983 | test | INFO | MainThread | [basetestcase:log_setup_status:538] ========= BaseTestCase setup started for test #3 test_concurrent_txn ========= 2020-10-02 05:27:34,525 | test | INFO | MainThread | [basetestcase:log_setup_status:538] ========= BaseTestCase setup started for test #3 test_concurrent_txn ========= 2020-10-02 05:27:34,750 | test | INFO | MainThread | [basetestcase:setUp:319] Initializing cluster 2020-10-02 05:27:34,970 | test | WARNING | MainThread | [basetestcase:_initialize_nodes:582] RAM quota was defined less than 100 MB: 2020-10-02 05:27:47,033 | test | INFO | MainThread | [basetestcase:setUp:341] Couchbase Cluster: default, Nodes: 172.23.105.229, 172.23.105.230 initialized 2020-10-02 05:27:47,046 | test | INFO | MainThread | [basetestcase:log_setup_status:538] ========= BaseTestCase setup finished for test #3 test_concurrent_txn ========= 2020-10-02 05:27:47,065 | test | INFO | MainThread | [basetestcase:log_setup_status:538] ========= CollectionBase setup started for test #3 test_concurrent_txn ========= 2020-10-02 05:27:49,471 | test | INFO | pool-4-thread-3 | [table_view:display:72] Rebalance Overview +----------------+-----------------+--------------+ | Nodes | Services | Status | +----------------+-----------------+--------------+ | 172.23.105.229 | index, kv, n1ql | Cluster node | | 172.23.105.230 | None | <--- IN --- | +----------------+-----------------+--------------+ 2020-10-02 05:27:49,483 | test | INFO | pool-4-thread-3 | [task:check:320] Rebalance - status: running, progress: 0 2020-10-02 05:27:59,520 | test | INFO | pool-4-thread-3 | [task:check:320] Rebalance - status: none, progress: 100 2020-10-02 05:27:59,532 | test | INFO | pool-4-thread-3 | [task:check:377] Rebalance completed with progress: 100% in 10.0610001087 sec 2020-10-02 05:27:59,686 | test | INFO | MainThread | [bucket_ready_functions:create_buckets_using_json_data:1754] Creating required buckets from template 2020-10-02 05:27:59,822 | test | INFO | Thread-101 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection yN47uRz-rs%kzusLNJp-F3N-m creation 2020-10-02 05:28:01,917 | test | INFO | Thread-103 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection jslC3HNIQ6U creation 2020-10-02 05:28:03,953 | test | INFO | Thread-103 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection NnQ3y_xLQhZnlJO6VNO creation 2020-10-02 05:28:06,030 | test | INFO | Thread-106 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection ktQFQQZ0lZpJwGd_5pkmCDBSFde creation 2020-10-02 05:28:08,066 | test | INFO | Thread-106 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection d9qWHq9GKA7rCiaOvTyo creation 2020-10-02 05:28:10,160 | test | INFO | Thread-109 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection W7UvsFZcgRrjxIq4ioL creation 2020-10-02 05:28:12,200 | test | INFO | Thread-109 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection hJa creation 2020-10-02 05:28:14,276 | test | INFO | Thread-112 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection L3A7rDT0DO8%rGiSz3 creation 2020-10-02 05:28:16,319 | test | INFO | Thread-112 | [common_lib:sleep:22] Sleep 2 seconds. Reason: MB-38073: Wait after collection WoPU4M creation 2020-10-02 05:28:18,321 | test | INFO | MainThread | [bucket_ready_functions:wait_for_collection_creation_to_complete:2215] Waiting for all collections to be created 2020-10-02 05:28:18,381 | test | INFO | MainThread | [table_view:display:72] Bucket statistics +--------+-----------+----------+------------+-----+-------+------------+----------+-----------+ | Bucket | Type | Replicas | Durability | TTL | Items | RAM Quota | RAM Used | Disk Used | +--------+-----------+----------+------------+-----+-------+------------+----------+-----------+ | UgD | couchbase | 1 | none | 0 | 0 | 5322571776 | 57005392 | 70997444 | +--------+-----------+----------+------------+-----+-------+------------+----------+-----------+ 2020-10-02 05:28:18,381 | test | INFO | MainThread | [collections_base:collection_setup:118] Creating required SDK clients for client_pool 2020-10-02 05:28:26,236 | test | INFO | MainThread | [common_lib:sleep:22] Sleep 120 seconds. Reason: MB-38497 2020-10-02 05:30:26,236 | test | INFO | MainThread | [bucket_ready_functions:perform_tasks_from_spec:4532] Performing scope/collection specific operations 2020-10-02 05:30:31,351 | test | INFO | MainThread | [table_view:display:72] Cluster statistics +----------------+-----------------+-----------------+------------+------------+------------------------+-------------------+-----------------------+ | Node | Services | CPU_utilization | Mem_total | Mem_free | Swap_mem_used | Active / Replica | Version | +----------------+-----------------+-----------------+------------+------------+------------------------+-------------------+-----------------------+ | 172.23.105.229 | index, kv, n1ql | 2.51889168766 | 4201840640 | 3442847744 | 112721920 / 3758092288 | 0 / 0 | 7.0.0-3264-enterprise | | 172.23.105.230 | kv | 2.01005025126 | 4201844736 | 3481980928 | 20709376 / 3758092288 | 0 / 0 | 7.0.0-3264-enterprise | +----------------+-----------------+-----------------+------------+------------+------------------------+-------------------+-----------------------+ 2020-10-02 05:30:32,648 | test | INFO | MainThread | [bucket_ready_functions:validate_docs_per_collections_all_buckets:4342] Validating collection stats and item counts 2020-10-02 05:30:39,081 | test | INFO | MainThread | [table_view:display:72] Bucket statistics +--------+-----------+----------+------------+-----+-------+------------+----------+-----------+ | Bucket | Type | Replicas | Durability | TTL | Items | RAM Quota | RAM Used | Disk Used | +--------+-----------+----------+------------+-----+-------+------------+----------+-----------+ | UgD | couchbase | 1 | none | 0 | 20000 | 5322571776 | 83285312 | 117567136 | +--------+-----------+----------+------------+-----+-------+------------+----------+-----------+ 2020-10-02 05:30:39,082 | test | INFO | MainThread | [basetestcase:log_setup_status:538] ========= CollectionBase setup complete for test #3 test_concurrent_txn ========= 2020-10-02 05:30:39,128 | test | INFO | MainThread | [N1qlBase:run_cbq_query:137] RUN QUERY CREATE PRIMARY INDEX ix1a on default:`UgD`.`-R`.`d9qWHq9GKA7rCiaOvTyo` USING GSI 2020-10-02 05:30:41,448 | test | INFO | MainThread | [N1qlBase:run_cbq_query:193] TOTAL ELAPSED TIME: 2.31540611s 2020-10-02 05:30:45,190 | test | INFO | MainThread | [N1qlBase:get_stmt_for_threads:760] insert, delete and update 4 0 6 2020-10-02 05:30:45,204 | test | INFO | MainThread | [N1qlBase:run_cbq_query:137] RUN QUERY BEGIN WORK 2020-10-02 05:30:45,210 | test | INFO | MainThread | [N1qlBase:run_cbq_query:193] TOTAL ELAPSED TIME: 1.631981ms 2020-10-02 05:30:45,232 | test | INFO | MainThread | [N1qlBase:run_cbq_query:137] RUN QUERY BEGIN WORK 2020-10-02 05:30:45,236 | test | INFO | MainThread | [N1qlBase:run_cbq_query:193] TOTAL ELAPSED TIME: 644.336µs 2020-10-02 05:30:45,249 | test | INFO | MainThread | [N1qlBase:run_cbq_query:137] RUN QUERY BEGIN WORK 2020-10-02 05:30:45,253 | test | INFO | MainThread | [N1qlBase:run_cbq_query:193] TOTAL ELAPSED TIME: 564.474µs 2020-10-02 05:30:45,263 | test | INFO | MainThread | [N1qlBase:run_cbq_query:137] RUN QUERY BEGIN WORK 2020-10-02 05:30:45,269 | test | INFO | MainThread | [N1qlBase:run_cbq_query:193] TOTAL ELAPSED TIME: 786.771µs 2020-10-02 05:30:45,269 | test | INFO | Thread-148 | [N1qlBase:thread_txn:742] values are ["UgD.-R.d9qWHq9GKA7rCiaOvTyo:DELETE: join_yr > 2016 OR job_title='Engineer':mutated=1", "UgD.-R.d9qWHq9GKA7rCiaOvTyo:DELETE: Working_country='USA':temp_emp=True", 'SAVEPOINT:a0', 'SAVEPOINT:a0', 'SAVEPOINT:a1', 'SAVEPOINT:a0', "UgD.-R.d9qWHq9GKA7rCiaOvTyo:INSERT: citizen_of='USA' AND job_title='Sales'"] True False False 2020-10-02 05:30:45,269 | test | INFO | Thread-149 | [N1qlBase:thread_txn:742] values are ['UgD.-R.d9qWHq9GKA7rCiaOvTyo:INSERT: join_mo=9', 'SAVEPOINT:a0', 'SAVEPOINT:a1', 'SAVEPOINT:a0', "UgD.-R.d9qWHq9GKA7rCiaOvTyo:DELETE: ['Development'] in skills:gaming=True", "UgD.-R.d9qWHq9GKA7rCiaOvTyo:DELETE: join_mo =1:join_month='January'", 'SAVEPOINT:a0'] True False False 2020-10-02 05:30:45,269 | test | INFO | Thread-150 | [N1qlBase:thread_txn:742] values are ['SAVEPOINT:a0', "UgD.-R.d9qWHq9GKA7rCiaOvTyo:DELETE: job_title='Engineer' and 'Testing' in skills:QE=True", 'SAVEPOINT:a0', "UgD.-R.d9qWHq9GKA7rCiaOvTyo:DELETE: job_title='Support':SE=True", 'SAVEPOINT:a0', 'SAVEPOINT:a1', "UgD.-R.d9qWHq9GKA7rCiaOvTyo:INSERT: job_title='Support'"] True False False 2020-10-02 05:30:45,269 | test | INFO | Thread-151 | [N1qlBase:thread_txn:742] values are ['SAVEPOINT:a1', 'SAVEPOINT:a0', "UgD.-R.d9qWHq9GKA7rCiaOvTyo:INSERT: citizen_of='INDIA'", 'SAVEPOINT:a1', 'SAVEPOINT:a1'] False False False 2020-10-02 05:30:45,286 | test | INFO | Thread-149 | [N1qlBase:run_cbq_query:137] RUN QUERY INSERT INTO default:`UgD`.`-R`.`d9qWHq9GKA7rCiaOvTyo` (KEY k1, value t) SELECT DISTINCT t.name AS k1,t FROM default:`UgD`.`-R`.`d9qWHq9GKA7rCiaOvTyo` t WHERE join_mo=9 LIMIT 10 RETURNING * 2020-10-02 05:30:45,289 | test | INFO | Thread-148 | [N1qlBase:run_cbq_query:137] RUN QUERY DELETE FROM default:`UgD`.`-R`.`d9qWHq9GKA7rCiaOvTyo` WHERE join_yr > 2016 OR job_title='Engineer' LIMIT 200 RETURNING meta().id 2020-10-02 05:30:45,289 | test | INFO | Thread-150 | [N1qlBase:run_cbq_query:137] RUN QUERY SAVEPOINT a0 2020-10-02 05:30:45,290 | test | INFO | Thread-151 | [N1qlBase:run_cbq_query:137] RUN QUERY SAVEPOINT a1 2020-10-02 05:30:45,293 | test | INFO | Thread-150 | [N1qlBase:run_cbq_query:193] TOTAL ELAPSED TIME: 445.869µs 2020-10-02 05:30:45,295 | test | INFO | Thread-151 | [N1qlBase:run_cbq_query:193] TOTAL ELAPSED TIME: 412.432µs 2020-10-02 05:30:45,305 | test | INFO | Thread-150 | [N1qlBase:run_cbq_query:137] RUN QUERY DELETE FROM default:`UgD`.`-R`.`d9qWHq9GKA7rCiaOvTyo` WHERE job_title='Engineer' and 'Testing' in skills LIMIT 200 RETURNING meta().id 2020-10-02 05:30:45,306 | test | INFO | Thread-151 | [N1qlBase:run_cbq_query:137] RUN QUERY SAVEPOINT a0 2020-10-02 05:30:45,312 | test | INFO | Thread-151 | [N1qlBase:run_cbq_query:193] TOTAL ELAPSED TIME: 883.865µs 2020-10-02 05:30:45,322 | test | INFO | Thread-151 | [N1qlBase:run_cbq_query:137] RUN QUERY INSERT INTO default:`UgD`.`-R`.`d9qWHq9GKA7rCiaOvTyo` (KEY k1, value t) SELECT DISTINCT t.name AS k1,t FROM default:`UgD`.`-R`.`d9qWHq9GKA7rCiaOvTyo` t WHERE citizen_of='INDIA' LIMIT 10 RETURNING * 2020-10-02 05:30:46,144 | test | INFO | Thread-149 | [N1qlBase:run_cbq_query:193] TOTAL ELAPSED TIME: 850.861733ms 2020-10-02 05:30:46,180 | test | INFO | Thread-149 | [N1qlBase:run_cbq_query:137] RUN QUERY SELECT * from default:`UgD`.`-R`.`d9qWHq9GKA7rCiaOvTyo` WHERE META().id in ['Basha White', 'Basha Wilson', 'Alvita Jackson', 'Basha Bailey', 'Basha Juan Jose', 'Alvita Cook', 'Alvita Baker', 'Alvita Moore', 'Alvita Palmer', 'Alvita Howard Jr.'] 2020-10-02 05:30:46,572 | test | INFO | Thread-149 | [N1qlBase:run_cbq_query:193] TOTAL ELAPSED TIME: 387.995867ms 2020-10-02 05:30:46,582 | test | INFO | Thread-149 | [N1qlBase:run_cbq_query:137] RUN QUERY SAVEPOINT a0 2020-10-02 05:30:46,586 | test | INFO | Thread-149 | [N1qlBase:run_cbq_query:193] TOTAL ELAPSED TIME: 574.441µs 2020-10-02 05:30:46,599 | test | INFO | Thread-149 | [N1qlBase:run_cbq_query:137] RUN QUERY SAVEPOINT a1 2020-10-02 05:30:46,604 | test | INFO | Thread-149 | [N1qlBase:run_cbq_query:193] TOTAL ELAPSED TIME: 669.79µs 2020-10-02 05:30:46,618 | test | INFO | Thread-149 | [N1qlBase:run_cbq_query:137] RUN QUERY SAVEPOINT a0 2020-10-02 05:30:46,621 | test | INFO | Thread-149 | [N1qlBase:run_cbq_query:193] TOTAL ELAPSED TIME: 544.059µs 2020-10-02 05:30:46,634 | test | INFO | Thread-149 | [N1qlBase:run_cbq_query:137] RUN QUERY DELETE FROM default:`UgD`.`-R`.`d9qWHq9GKA7rCiaOvTyo` WHERE ['Development'] in skills LIMIT 200 RETURNING meta().id 2020-10-02 05:30:46,650 | test | INFO | Thread-148 | [N1qlBase:run_cbq_query:193] TOTAL ELAPSED TIME: 1.356845332s 2020-10-02 05:30:46,653 | test | INFO | Thread-151 | [N1qlBase:run_cbq_query:193] TOTAL ELAPSED TIME: 1.324924202s 2020-10-02 05:30:46,657 | test | INFO | Thread-150 | [N1qlBase:run_cbq_query:193] TOTAL ELAPSED TIME: 1.347169121s 2020-10-02 05:30:46,663 | test | INFO | Thread-148 | [N1qlBase:run_cbq_query:137] RUN QUERY SELECT META().id,* from default:`UgD`.`-R`.`d9qWHq9GKA7rCiaOvTyo` WHERE META().id in [] 2020-10-02 05:30:46,667 | test | INFO | Thread-151 | [N1qlBase:run_cbq_query:137] RUN QUERY SELECT * from default:`UgD`.`-R`.`d9qWHq9GKA7rCiaOvTyo` WHERE META().id in [] 2020-10-02 05:30:46,670 | test | INFO | Thread-150 | [N1qlBase:run_cbq_query:137] RUN QUERY SELECT META().id,* from default:`UgD`.`-R`.`d9qWHq9GKA7rCiaOvTyo` WHERE META().id in [] 2020-10-02 05:30:47,355 | test | INFO | Thread-151 | [N1qlBase:run_cbq_query:193] TOTAL ELAPSED TIME: 684.13592ms 2020-10-02 05:30:47,361 | test | INFO | Thread-148 | [N1qlBase:run_cbq_query:193] TOTAL ELAPSED TIME: 692.324373ms 2020-10-02 05:30:47,361 | test | INFO | Thread-148 | [N1qlBase:validate_delete_results:380] delete result is [] 2020-10-02 05:30:47,367 | test | INFO | Thread-151 | [N1qlBase:run_cbq_query:137] RUN QUERY SAVEPOINT a1 2020-10-02 05:30:47,371 | test | INFO | Thread-148 | [N1qlBase:run_cbq_query:137] RUN QUERY DELETE FROM default:`UgD`.`-R`.`d9qWHq9GKA7rCiaOvTyo` WHERE Working_country='USA' LIMIT 200 RETURNING meta().id 2020-10-02 05:30:47,372 | test | INFO | Thread-151 | [N1qlBase:run_cbq_query:193] TOTAL ELAPSED TIME: 543.816µs 2020-10-02 05:30:47,414 | test | INFO | Thread-150 | [N1qlBase:run_cbq_query:193] TOTAL ELAPSED TIME: 701.286902ms 2020-10-02 05:30:47,415 | test | INFO | Thread-150 | [N1qlBase:validate_delete_results:380] delete result is [] 2020-10-02 05:30:47,427 | test | INFO | Thread-151 | [N1qlBase:run_cbq_query:137] RUN QUERY SAVEPOINT a1 2020-10-02 05:30:47,430 | test | INFO | Thread-150 | [N1qlBase:run_cbq_query:137] RUN QUERY SAVEPOINT a0 2020-10-02 05:30:47,431 | test | INFO | Thread-151 | [N1qlBase:run_cbq_query:193] TOTAL ELAPSED TIME: 434.788µs 2020-10-02 05:30:47,431 | test | INFO | Thread-151 | [N1qlBase:end_txn:94] commit work for txn {'txid': u'165f77da-35b7-4e8a-bf16-58e9692a2cc4', 'scan_consistency': 'REQUEST_PLUS'} 2020-10-02 05:30:47,434 | test | INFO | Thread-150 | [N1qlBase:run_cbq_query:193] TOTAL ELAPSED TIME: 356.096µs 2020-10-02 05:30:47,441 | test | INFO | Thread-151 | [N1qlBase:run_cbq_query:137] RUN QUERY ROLLBACK WORK 2020-10-02 05:30:47,446 | test | INFO | Thread-150 | [N1qlBase:run_cbq_query:137] RUN QUERY DELETE FROM default:`UgD`.`-R`.`d9qWHq9GKA7rCiaOvTyo` WHERE job_title='Support' LIMIT 200 RETURNING meta().id 2020-10-02 05:30:47,447 | test | INFO | Thread-151 | [N1qlBase:run_cbq_query:193] TOTAL ELAPSED TIME: 603.672µs 2020-10-02 05:30:47,447 | test | INFO | Thread-151 | [N1qlBase:thread_txn:748] queries executed in txn are {u'165f77da-35b7-4e8a-bf16-58e9692a2cc4': [{u'signature': u'json', u'requestID': u'c42d4570-ae22-41b3-ba79-f533d2c12d78', u'metrics': {u'resultCount': 0, u'executionTime': u'323.491\xb5s', u'serviceLoad': 4, u'resultSize': 0, u'elapsedTime': u'412.432\xb5s'}, u'results': [], u'status': u'success'}, {u'signature': u'json', u'requestID': u'ab585082-9b4f-4820-856e-3b4faebaf338', u'metrics': {u'resultCount': 0, u'executionTime': u'739.727\xb5s', u'serviceLoad': 6, u'resultSize': 0, u'elapsedTime': u'883.865\xb5s'}, u'results': [], u'status': u'success'}, "INSERT INTO default:`UgD`.`-R`.`d9qWHq9GKA7rCiaOvTyo` (KEY k1, value t) SELECT DISTINCT t.name AS k1,t FROM default:`UgD`.`-R`.`d9qWHq9GKA7rCiaOvTyo` t WHERE citizen_of='INDIA' LIMIT 10 RETURNING *", {u'signature': u'json', u'requestID': u'10057a43-bc35-43b9-8904-43992cbd6678', u'metrics': {u'resultCount': 0, u'executionTime': u'445.682\xb5s', u'serviceLoad': 4, u'resultSize': 0, u'elapsedTime': u'543.816\xb5s'}, u'results': [], u'status': u'success'}, {u'signature': u'json', u'requestID': u'917b832e-eb35-494d-81fc-44a1a6b13cbe', u'metrics': {u'resultCount': 0, u'executionTime': u'353.078\xb5s', u'serviceLoad': 4, u'resultSize': 0, u'elapsedTime': u'434.788\xb5s'}, u'results': [], u'status': u'success'}, 'ROLLBACK WORK']} 2020-10-02 05:30:47,875 | test | INFO | Thread-149 | [N1qlBase:run_cbq_query:193] TOTAL ELAPSED TIME: 1.23522809s 2020-10-02 05:30:47,885 | test | INFO | Thread-149 | [N1qlBase:run_cbq_query:137] RUN QUERY SELECT META().id,* from default:`UgD`.`-R`.`d9qWHq9GKA7rCiaOvTyo` WHERE META().id in [] 2020-10-02 05:30:47,901 | test | INFO | Thread-148 | [N1qlBase:run_cbq_query:193] TOTAL ELAPSED TIME: 524.09809ms 2020-10-02 05:30:47,915 | test | INFO | Thread-148 | [N1qlBase:run_cbq_query:137] RUN QUERY SELECT META().id,* from default:`UgD`.`-R`.`d9qWHq9GKA7rCiaOvTyo` WHERE META().id in ['test_collections-1000', 'test_collections-1008', 'test_collections-1011', 'test_collections-1013', 'test_collections-1015', 'test_collections-1020', 'test_collections-1027', 'test_collections-1032', 'test_collections-1047', 'test_collections-1048', 'test_collections-105', 'test_collections-1050', 'test_collections-1053', 'test_collections-1061', 'test_collections-1063', 'test_collections-107', 'test_collections-1073', 'test_collections-1075', 'test_collections-1076', 'test_collections-1080', 'test_collections-1084', 'test_collections-1085', 'test_collections-1086', 'test_collections-1088', 'test_collections-1097', 'test_collections-1104', 'test_collections-1106', 'test_collections-1109', 'test_collections-111', 'test_collections-1113', 'test_collections-1115', 'test_collections-1122', 'test_collections-1124', 'test_collections-1127', 'test_collections-1130', 'test_collections-1133', 'test_collections-114', 'test_collections-1142', 'test_collections-1145', 'test_collections-1151', 'test_collections-1152', 'test_collections-1153', 'test_collections-1159', 'test_collections-116', 'test_collections-1164', 'test_collections-1166', 'test_collections-1169', 'test_collections-1170', 'test_collections-1182', 'test_collections-1185', 'test_collections-1186', 'test_collections-119', 'test_collections-1195', 'test_collections-1198', 'test_collections-1205', 'test_collections-1209', 'test_collections-1216', 'test_collections-1222', 'test_collections-1225', 'test_collections-123', 'test_collections-1231', 'test_collections-1240', 'test_collections-1252', 'test_collections-1258', 'test_collections-1261', 'test_collections-1263', 'test_collections-1267', 'test_collections-1270', 'test_collections-1271', 'test_collections-1276', 'test_collections-1279', 'test_collections-128', 'test_collections-1280', 'test_collections-1287', 'test_collections-1289', 'test_collections-1290', 'test_collections-1292', 'test_collections-1294', 'test_collections-1297', 'test_collections-1299', 'test_collections-13', 'test_collections-1302', 'test_collections-131', 'test_collections-1310', 'test_collections-1312', 'test_collections-1314', 'test_collections-1322', 'test_collections-1323', 'test_collections-1330', 'test_collections-1333', 'test_collections-1339', 'test_collections-1341', 'test_collections-1344', 'test_collections-1348', 'test_collections-1350', 'test_collections-1357', 'test_collections-1359', 'test_collections-136', 'test_collections-1360', 'test_collections-1364', 'test_collections-1369', 'test_collections-1379', 'test_collections-1382', 'test_collections-1384', 'test_collections-1385', 'test_collections-1395', 'test_collections-14', 'test_collections-1402', 'test_collections-1403', 'test_collections-1405', 'test_collections-1410', 'test_collections-1416', 'test_collections-1419', 'test_collections-1426', 'test_collections-1429', 'test_collections-143', 'test_collections-1436', 'test_collections-1440', 'test_collections-1444', 'test_collections-1446', 'test_collections-1449', 'test_collections-1453', 'test_collections-1455', 'test_collections-1456', 'test_collections-146', 'test_collections-1462', 'test_collections-1465', 'test_collections-147', 'test_collections-1475', 'test_collections-1476', 'test_collections-1481', 'test_collections-1482', 'test_collections-1485', 'test_collections-1492', 'test_collections-150', 'test_collections-1500', 'test_collections-1501', 'test_collections-1503', 'test_collections-1505', 'test_collections-1507', 'test_collections-1508', 'test_collections-1509', 'test_collections-1512', 'test_collections-1514', 'test_collections-1527', 'test_collections-1530', 'test_collections-1535', 'test_collections-1539', 'test_collections-154', 'test_collections-1542', 'test_collections-1543', 'test_collections-1549', 'test_collections-155', 'test_collections-1551', 'test_collections-1552', 'test_collections-1553', 'test_collections-1561', 'test_collections-1578', 'test_collections-1588', 'test_collections-1597', 'test_collections-1598', 'test_collections-1601', 'test_collections-1604', 'test_collections-1607', 'test_collections-1612', 'test_collections-1614', 'test_collections-1619', 'test_collections-1622', 'test_collections-1623', 'test_collections-1624', 'test_collections-1627', 'test_collections-1630', 'test_collections-1631', 'test_collections-1634', 'test_collections-1636', 'test_collections-1641', 'test_collections-1643', 'test_collections-1645', 'test_collections-165', 'test_collections-1650', 'test_collections-1653', 'test_collections-1654', 'test_collections-166', 'test_collections-1660', 'test_collections-1666', 'test_collections-167', 'test_collections-1674', 'test_collections-1676', 'test_collections-1684', 'test_collections-1685', 'test_collections-1686', 'test_collections-169', 'test_collections-1693', 'test_collections-1695', 'test_collections-1696', 'test_collections-1697', 'test_collections-1699', 'test_collections-1703', 'test_collections-1710', 'test_collections-1715'] 2020-10-02 05:30:48,265 | test | INFO | Thread-149 | [N1qlBase:run_cbq_query:193] TOTAL ELAPSED TIME: 373.966684ms 2020-10-02 05:30:48,265 | test | INFO | Thread-149 | [N1qlBase:validate_delete_results:380] delete result is [] 2020-10-02 05:30:48,279 | test | INFO | Thread-149 | [N1qlBase:run_cbq_query:137] RUN QUERY DELETE FROM default:`UgD`.`-R`.`d9qWHq9GKA7rCiaOvTyo` WHERE join_mo =1 LIMIT 200 RETURNING meta().id 2020-10-02 05:30:48,384 | test | INFO | Thread-148 | [N1qlBase:run_cbq_query:193] TOTAL ELAPSED TIME: 462.713803ms 2020-10-02 05:30:48,384 | test | INFO | Thread-148 | [N1qlBase:validate_delete_results:380] delete result is [] 2020-10-02 05:30:48,394 | test | INFO | Thread-148 | [N1qlBase:run_cbq_query:137] RUN QUERY SAVEPOINT a0 2020-10-02 05:30:48,398 | test | INFO | Thread-148 | [N1qlBase:run_cbq_query:193] TOTAL ELAPSED TIME: 600.56µs 2020-10-02 05:30:48,411 | test | INFO | Thread-148 | [N1qlBase:run_cbq_query:137] RUN QUERY SAVEPOINT a0 2020-10-02 05:30:48,415 | test | INFO | Thread-148 | [N1qlBase:run_cbq_query:193] TOTAL ELAPSED TIME: 598.534µs 2020-10-02 05:30:48,426 | test | INFO | Thread-150 | [N1qlBase:run_cbq_query:193] TOTAL ELAPSED TIME: 975.312878ms 2020-10-02 05:30:48,428 | test | INFO | Thread-148 | [N1qlBase:run_cbq_query:137] RUN QUERY SAVEPOINT a1 2020-10-02 05:30:48,434 | test | INFO | Thread-148 | [N1qlBase:run_cbq_query:193] TOTAL ELAPSED TIME: 423.934µs 2020-10-02 05:30:48,437 | test | INFO | Thread-150 | [N1qlBase:run_cbq_query:137] RUN QUERY SELECT META().id,* from default:`UgD`.`-R`.`d9qWHq9GKA7rCiaOvTyo` WHERE META().id in [] 2020-10-02 05:30:48,444 | test | INFO | Thread-148 | [N1qlBase:run_cbq_query:137] RUN QUERY SAVEPOINT a0 2020-10-02 05:30:48,448 | test | INFO | Thread-148 | [N1qlBase:run_cbq_query:193] TOTAL ELAPSED TIME: 472.175µs 2020-10-02 05:30:48,461 | test | INFO | Thread-148 | [N1qlBase:run_cbq_query:137] RUN QUERY INSERT INTO default:`UgD`.`-R`.`d9qWHq9GKA7rCiaOvTyo` (KEY k1, value t) SELECT DISTINCT t.name AS k1,t FROM default:`UgD`.`-R`.`d9qWHq9GKA7rCiaOvTyo` t WHERE citizen_of='USA' AND job_title='Sales' LIMIT 10 RETURNING * 2020-10-02 05:30:48,868 | test | INFO | Thread-150 | [N1qlBase:run_cbq_query:193] TOTAL ELAPSED TIME: 426.816084ms 2020-10-02 05:30:48,868 | test | INFO | Thread-150 | [N1qlBase:validate_delete_results:380] delete result is [] 2020-10-02 05:30:48,878 | test | INFO | Thread-150 | [N1qlBase:run_cbq_query:137] RUN QUERY SAVEPOINT a0 2020-10-02 05:30:48,884 | test | INFO | Thread-150 | [N1qlBase:run_cbq_query:193] TOTAL ELAPSED TIME: 456.55µs 2020-10-02 05:30:48,894 | test | INFO | Thread-150 | [N1qlBase:run_cbq_query:137] RUN QUERY SAVEPOINT a1 2020-10-02 05:30:48,901 | test | INFO | Thread-150 | [N1qlBase:run_cbq_query:193] TOTAL ELAPSED TIME: 472.299µs 2020-10-02 05:30:48,911 | test | INFO | Thread-150 | [N1qlBase:run_cbq_query:137] RUN QUERY INSERT INTO default:`UgD`.`-R`.`d9qWHq9GKA7rCiaOvTyo` (KEY k1, value t) SELECT DISTINCT t.name AS k1,t FROM default:`UgD`.`-R`.`d9qWHq9GKA7rCiaOvTyo` t WHERE job_title='Support' LIMIT 10 RETURNING * 2020-10-02 05:30:49,206 | test | INFO | Thread-149 | [N1qlBase:run_cbq_query:193] TOTAL ELAPSED TIME: 921.192416ms 2020-10-02 05:30:49,227 | test | INFO | Thread-149 | [N1qlBase:run_cbq_query:137] RUN QUERY SELECT META().id,* from default:`UgD`.`-R`.`d9qWHq9GKA7rCiaOvTyo` WHERE META().id in ['test_collections-10', 'test_collections-1027', 'test_collections-1034', 'test_collections-1037', 'test_collections-1041', 'test_collections-1048', 'test_collections-1062', 'test_collections-1065', 'test_collections-1076', 'test_collections-1087', 'test_collections-1127', 'test_collections-1134', 'test_collections-1155', 'test_collections-1162', 'test_collections-1165', 'test_collections-1169', 'test_collections-1176', 'test_collections-1190', 'test_collections-1193', 'test_collections-1204', 'test_collections-1215', 'test_collections-1262', 'test_collections-1283', 'test_collections-1290', 'test_collections-1293', 'test_collections-1297', 'test_collections-13', 'test_collections-1304', 'test_collections-131', 'test_collections-1318', 'test_collections-1321', 'test_collections-1332', 'test_collections-1343', 'test_collections-1360', 'test_collections-138', 'test_collections-1390', 'test_collections-141', 'test_collections-1411', 'test_collections-1418', 'test_collections-1421', 'test_collections-1425', 'test_collections-1432', 'test_collections-1446', 'test_collections-1449', 'test_collections-1460', 'test_collections-1471', 'test_collections-1488', 'test_collections-1518', 'test_collections-152', 'test_collections-1539', 'test_collections-1546', 'test_collections-1549', 'test_collections-1553', 'test_collections-1560', 'test_collections-1574', 'test_collections-1588', 'test_collections-1599', 'test_collections-1639', 'test_collections-1646', 'test_collections-166', 'test_collections-1660', 'test_collections-1667', 'test_collections-1674', 'test_collections-1677', 'test_collections-1681', 'test_collections-1688', 'test_collections-169', 'test_collections-17', 'test_collections-1702', 'test_collections-1716', 'test_collections-1727', 'test_collections-1767', 'test_collections-1774', 'test_collections-1795', 'test_collections-180', 'test_collections-1802', 'test_collections-1805', 'test_collections-1809', 'test_collections-1816', 'test_collections-1830', 'test_collections-1833', 'test_collections-1844', 'test_collections-1855', 'test_collections-1895', 'test_collections-1902', 'test_collections-191', 'test_collections-1923', 'test_collections-1930', 'test_collections-1933', 'test_collections-1937', 'test_collections-1944', 'test_collections-1958', 'test_collections-1961', 'test_collections-1972', 'test_collections-1983', 'test_collections-208', 'test_collections-219', 'test_collections-24', 'test_collections-259', 'test_collections-266', 'test_collections-269', 'test_collections-280', 'test_collections-294', 'test_collections-297', 'test_collections-3', 'test_collections-308', 'test_collections-319', 'test_collections-336', 'test_collections-347', 'test_collections-38', 'test_collections-387', 'test_collections-394', 'test_collections-397', 'test_collections-408', 'test_collections-41', 'test_collections-422', 'test_collections-425', 'test_collections-436', 'test_collections-447', 'test_collections-450', 'test_collections-464', 'test_collections-475', 'test_collections-515', 'test_collections-52', 'test_collections-522', 'test_collections-525', 'test_collections-529', 'test_collections-536', 'test_collections-550', 'test_collections-553', 'test_collections-564', 'test_collections-575', 'test_collections-592', 'test_collections-622', 'test_collections-63', 'test_collections-643', 'test_collections-650', 'test_collections-653', 'test_collections-657', 'test_collections-664', 'test_collections-678', 'test_collections-681', 'test_collections-692', 'test_collections-703', 'test_collections-720', 'test_collections-750', 'test_collections-771', 'test_collections-778', 'test_collections-781', 'test_collections-785', 'test_collections-792', 'test_collections-80', 'test_collections-806', 'test_collections-809', 'test_collections-820', 'test_collections-831', 'test_collections-848', 'test_collections-899', 'test_collections-906', 'test_collections-909', 'test_collections-91', 'test_collections-913', 'test_collections-920', 'test_collections-934', 'test_collections-937', 'test_collections-948', 'test_collections-959', 'test_collections-976', 'test_collections-987'] 2020-10-02 05:30:49,348 | test | INFO | Thread-148 | [N1qlBase:run_cbq_query:193] TOTAL ELAPSED TIME: 881.397483ms 2020-10-02 05:30:49,358 | test | INFO | Thread-148 | [N1qlBase:run_cbq_query:137] RUN QUERY SELECT * from default:`UgD`.`-R`.`d9qWHq9GKA7rCiaOvTyo` WHERE META().id in [] 2020-10-02 05:30:49,565 | test | INFO | Thread-149 | [N1qlBase:run_cbq_query:193] TOTAL ELAPSED TIME: 332.03523ms 2020-10-02 05:30:49,565 | test | INFO | Thread-149 | [N1qlBase:validate_delete_results:380] delete result is [] 2020-10-02 05:30:49,576 | test | INFO | Thread-149 | [N1qlBase:run_cbq_query:137] RUN QUERY SAVEPOINT a0 2020-10-02 05:30:49,582 | test | INFO | Thread-149 | [N1qlBase:run_cbq_query:193] TOTAL ELAPSED TIME: 503.805µs 2020-10-02 05:30:49,605 | test | INFO | Thread-149 | [N1qlBase:run_cbq_query:137] RUN QUERY SELECT * FROM system:transactions 2020-10-02 05:30:49,618 | test | INFO | Thread-149 | [N1qlBase:run_cbq_query:193] TOTAL ELAPSED TIME: 6.839388ms 2020-10-02 05:30:49,619 | test | INFO | Thread-149 | [N1qlBase:end_txn:94] commit work for txn {'txid': u'58915dd1-343e-46af-8368-178efdd71f62', 'scan_consistency': 'REQUEST_PLUS'} 2020-10-02 05:30:49,631 | test | INFO | Thread-149 | [N1qlBase:run_cbq_query:137] RUN QUERY COMMIT WORK 2020-10-02 05:30:49,750 | test | INFO | Thread-148 | [N1qlBase:run_cbq_query:193] TOTAL ELAPSED TIME: 387.487745ms 2020-10-02 05:30:49,760 | test | INFO | Thread-148 | [N1qlBase:run_cbq_query:137] RUN QUERY SELECT * FROM system:transactions 2020-10-02 05:30:49,767 | test | INFO | Thread-148 | [N1qlBase:run_cbq_query:193] TOTAL ELAPSED TIME: 1.13874ms 2020-10-02 05:30:49,767 | test | INFO | Thread-148 | [N1qlBase:end_txn:94] commit work for txn {'txid': u'1355853e-2ee4-49f3-8292-d0085e2d986c', 'scan_consistency': 'REQUEST_PLUS'} 2020-10-02 05:30:49,779 | test | INFO | Thread-148 | [N1qlBase:run_cbq_query:137] RUN QUERY COMMIT WORK 2020-10-02 05:30:49,905 | test | INFO | Thread-148 | [N1qlBase:run_cbq_query:193] TOTAL ELAPSED TIME: 121.348443ms 2020-10-02 05:30:49,905 | test | INFO | Thread-148 | [N1qlBase:run_cbq_query:196] txn failed 2020-10-02 05:30:49,905 | test | INFO | Thread-148 | [N1qlBase:run_cbq_query:205] txn failed with error {u'signature': u'json', u'requestID': u'88bdeb46-e0b9-4a14-a85e-b2200519316d', u'metrics': {u'resultCount': 0, u'executionTime': u'121.260359ms', u'serviceLoad': 4, u'resultSize': 0, u'errorCount': 1, u'elapsedTime': u'121.348443ms'}, u'results': [], u'errors': [{u'msg': u'Commit Transaction statement error: transaction operation failed | document exists | {"status_code":2,"bucket":"UgD","scope":"-R","collection":"d9qWHq9GKA7rCiaOvTyo","collection_id":12,"error_name":"KEY_EEXISTS","error_description":"key already exists, or CAS mismatch","opaque":943,"last_dispatched_to":"172.23.105.230:11210","last_dispatched_from":"172.23.105.229:32920","last_connection_id":"7953c2b31542e2f6/3cb2cd619b724063"}', u'code': 17007}], u'status': u'errors'} 2020-10-02 05:30:49,907 | test | INFO | Thread-148 | [N1qlBase:validate_error_during_commit:668] query is SELECT META().id,* from default:`UgD`.`-R`.`d9qWHq9GKA7rCiaOvTyo` WHERE META().id in [] 2020-10-02 05:30:49,918 | test | INFO | Thread-148 | [N1qlBase:run_cbq_query:137] RUN QUERY SELECT META().id,* from default:`UgD`.`-R`.`d9qWHq9GKA7rCiaOvTyo` WHERE META().id in [] 2020-10-02 05:30:49,963 | test | INFO | Thread-148 | [N1qlBase:run_cbq_query:193] TOTAL ELAPSED TIME: 40.271323ms 2020-10-02 05:30:49,963 | test | INFO | Thread-148 | [N1qlBase:validate_error_during_commit:693] commit fails with CAS MISMATCH error retry the txn 2020-10-02 05:30:49,963 | test | INFO | Thread-148 | [N1qlBase:thread_txn:748] queries executed in txn are {u'1355853e-2ee4-49f3-8292-d0085e2d986c': ["DELETE FROM default:`UgD`.`-R`.`d9qWHq9GKA7rCiaOvTyo` WHERE join_yr > 2016 OR job_title='Engineer' LIMIT 200 RETURNING meta().id", "DELETE FROM default:`UgD`.`-R`.`d9qWHq9GKA7rCiaOvTyo` WHERE Working_country='USA' LIMIT 200 RETURNING meta().id", {u'signature': u'json', u'requestID': u'c4865339-aa25-467c-9e07-8c5ea45f1255', u'metrics': {u'resultCount': 0, u'executionTime': u'478.324\xb5s', u'serviceLoad': 4, u'resultSize': 0, u'elapsedTime': u'600.56\xb5s'}, u'results': [], u'status': u'success'}, {u'signature': u'json', u'requestID': u'04114ee2-cfdc-47f2-bfa1-3384a599bc4f', u'metrics': {u'resultCount': 0, u'executionTime': u'488.555\xb5s', u'serviceLoad': 4, u'resultSize': 0, u'elapsedTime': u'598.534\xb5s'}, u'results': [], u'status': u'success'}, {u'signature': u'json', u'requestID': u'1eed7de7-b9ca-4683-a265-7b2d1063c3f3', u'metrics': {u'resultCount': 0, u'executionTime': u'342.396\xb5s', u'serviceLoad': 3, u'resultSize': 0, u'elapsedTime': u'423.934\xb5s'}, u'results': [], u'status': u'success'}, {u'signature': u'json', u'requestID': u'7ae0a07d-a40e-48ff-9d5e-85a999f9702f', u'metrics': {u'resultCount': 0, u'executionTime': u'390.427\xb5s', u'serviceLoad': 4, u'resultSize': 0, u'elapsedTime': u'472.175\xb5s'}, u'results': [], u'status': u'success'}, "INSERT INTO default:`UgD`.`-R`.`d9qWHq9GKA7rCiaOvTyo` (KEY k1, value t) SELECT DISTINCT t.name AS k1,t FROM default:`UgD`.`-R`.`d9qWHq9GKA7rCiaOvTyo` t WHERE citizen_of='USA' AND job_title='Sales' LIMIT 10 RETURNING *", 'COMMIT WORK']} 2020-10-02 05:30:50,130 | test | ERROR | Thread-150 | [rest_client:_http_request:747] POST http://172.23.105.229:8093/query?txid=e8ddcf04-4f66-4a6f-8425-bc536e1a1c73&scan_consistency=REQUEST_PLUS&statement=INSERT+INTO+default%3A%60UgD%60.%60-R%60.%60d9qWHq9GKA7rCiaOvTyo%60+%28KEY+k1%2C+value+t%29+SELECT+DISTINCT+t.name+AS+k1%2Ct+FROM+default%3A%60UgD%60.%60-R%60.%60d9qWHq9GKA7rCiaOvTyo%60+t+WHERE++job_title%3D%27Support%27+LIMIT+10+RETURNING+%2A body: headers: {'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 500 reason: unknown { "requestID": "e354d8af-9d12-4f59-a5aa-fd6e8a31eaec", "signature": {"*":"*"}, "results": [ ], "errors": [{"code":5000,"msg":"txFetch - cause: transaction operation failed | previous operation failed"}], "status": "errors", "metrics": {"elapsedTime": "1.213762152s","executionTime": "1.213616562s","resultCount": 0,"resultSize": 0,"serviceLoad": 3,"errorCount": 1} } auth: Administrator:password 2020-10-02 05:30:50,131 | test | INFO | Thread-150 | [N1qlBase:run_cbq_query:193] TOTAL ELAPSED TIME: 1.213762152s 2020-10-02 05:30:50,131 | test | INFO | Thread-150 | [N1qlBase:run_cbq_query:196] txn failed 2020-10-02 05:30:50,132 | test | INFO | Thread-150 | [N1qlBase:full_execute_query:575] txn failed with unexpected errors {u'signature': {u'*': u'*'}, u'requestID': u'e354d8af-9d12-4f59-a5aa-fd6e8a31eaec', u'metrics': {u'resultCount': 0, u'executionTime': u'1.213616562s', u'serviceLoad': 3, u'resultSize': 0, u'errorCount': 1, u'elapsedTime': u'1.213762152s'}, u'results': [], u'errors': [{u'msg': u'txFetch - cause: transaction operation failed | previous operation failed', u'code': 5000}], u'status': u'errors'} 2020-10-02 05:30:50,132 | test | INFO | Thread-150 | [N1qlBase:thread_txn:748] queries executed in txn are {u'e8ddcf04-4f66-4a6f-8425-bc536e1a1c73': [{u'signature': u'json', u'requestID': u'eb7882f6-c883-4b9b-aba5-d6c2ac47674e', u'metrics': {u'resultCount': 0, u'executionTime': u'385.833\xb5s', u'serviceLoad': 4, u'resultSize': 0, u'elapsedTime': u'445.869\xb5s'}, u'results': [], u'status': u'success'}, "DELETE FROM default:`UgD`.`-R`.`d9qWHq9GKA7rCiaOvTyo` WHERE job_title='Engineer' and 'Testing' in skills LIMIT 200 RETURNING meta().id", {u'signature': u'json', u'requestID': u'6284e110-76a5-4916-8156-99c2a8919138', u'metrics': {u'resultCount': 0, u'executionTime': u'295.29\xb5s', u'serviceLoad': 4, u'resultSize': 0, u'elapsedTime': u'356.096\xb5s'}, u'results': [], u'status': u'success'}, "DELETE FROM default:`UgD`.`-R`.`d9qWHq9GKA7rCiaOvTyo` WHERE job_title='Support' LIMIT 200 RETURNING meta().id", {u'signature': u'json', u'requestID': u'bf7c8ef6-9d7b-4324-a736-2537d1e6c117', u'metrics': {u'resultCount': 0, u'executionTime': u'378.191\xb5s', u'serviceLoad': 4, u'resultSize': 0, u'elapsedTime': u'456.55\xb5s'}, u'results': [], u'status': u'success'}, {u'signature': u'json', u'requestID': u'a2b2f708-6899-497d-a2ce-06afe1fb4625', u'metrics': {u'resultCount': 0, u'executionTime': u'385.222\xb5s', u'serviceLoad': 4, u'resultSize': 0, u'elapsedTime': u'472.299\xb5s'}, u'results': [], u'status': u'success'}]} 2020-10-02 05:30:50,187 | test | INFO | Thread-149 | [N1qlBase:run_cbq_query:193] TOTAL ELAPSED TIME: 553.189051ms 2020-10-02 05:30:50,188 | test | INFO | Thread-149 | [N1qlBase:thread_txn:748] queries executed in txn are {u'58915dd1-343e-46af-8368-178efdd71f62': ['INSERT INTO default:`UgD`.`-R`.`d9qWHq9GKA7rCiaOvTyo` (KEY k1, value t) SELECT DISTINCT t.name AS k1,t FROM default:`UgD`.`-R`.`d9qWHq9GKA7rCiaOvTyo` t WHERE join_mo=9 LIMIT 10 RETURNING *', {u'signature': u'json', u'requestID': u'7f66e934-1603-4f15-822c-bd0146ada783', u'metrics': {u'resultCount': 0, u'executionTime': u'437.509\xb5s', u'serviceLoad': 6, u'resultSize': 0, u'elapsedTime': u'574.441\xb5s'}, u'results': [], u'status': u'success'}, {u'signature': u'json', u'requestID': u'87286820-87e3-40f2-8994-08eef86f2953', u'metrics': {u'resultCount': 0, u'executionTime': u'557.952\xb5s', u'serviceLoad': 6, u'resultSize': 0, u'elapsedTime': u'669.79\xb5s'}, u'results': [], u'status': u'success'}, {u'signature': u'json', u'requestID': u'df107efe-1d9b-4ea5-9477-adbf77acbbb2', u'metrics': {u'resultCount': 0, u'executionTime': u'465.072\xb5s', u'serviceLoad': 6, u'resultSize': 0, u'elapsedTime': u'544.059\xb5s'}, u'results': [], u'status': u'success'}, "DELETE FROM default:`UgD`.`-R`.`d9qWHq9GKA7rCiaOvTyo` WHERE ['Development'] in skills LIMIT 200 RETURNING meta().id", 'DELETE FROM default:`UgD`.`-R`.`d9qWHq9GKA7rCiaOvTyo` WHERE join_mo =1 LIMIT 200 RETURNING meta().id', {u'signature': u'json', u'requestID': u'b5d834ce-7081-46a1-bb07-2f29fb4f62b6', u'metrics': {u'resultCount': 0, u'executionTime': u'419.237\xb5s', u'serviceLoad': 4, u'resultSize': 0, u'elapsedTime': u'503.805\xb5s'}, u'results': [], u'status': u'success'}, 'COMMIT WORK']} 2020-10-02 05:30:50,188 | test | INFO | MainThread | [N1qlBase:get_stmt_for_threads:792] One of the thread has failed 2020-10-02 05:30:50,190 | test | INFO | MainThread | [basic_ops:test_concurrent_txn:46] result is [[{}, []], [{}, []], [{'a0:3': {'UgD.-R.d9qWHq9GKA7rCiaOvTyo': {'DELETE': ['test_collections-10', 'test_collections-1027', 'test_collections-1034', 'test_collections-1037', 'test_collections-1041', 'test_collections-1048', 'test_collections-1062', 'test_collections-1065', 'test_collections-1076', 'test_collections-1087', 'test_collections-1127', 'test_collections-1134', 'test_collections-1155', 'test_collections-1162', 'test_collections-1165', 'test_collections-1169', 'test_collections-1176', 'test_collections-1190', 'test_collections-1193', 'test_collections-1204', 'test_collections-1215', 'test_collections-1262', 'test_collections-1283', 'test_collections-1290', 'test_collections-1293', 'test_collections-1297', 'test_collections-13', 'test_collections-1304', 'test_collections-131', 'test_collections-1318', 'test_collections-1321', 'test_collections-1332', 'test_collections-1343', 'test_collections-1360', 'test_collections-138', 'test_collections-1390', 'test_collections-141', 'test_collections-1411', 'test_collections-1418', 'test_collections-1421', 'test_collections-1425', 'test_collections-1432', 'test_collections-1446', 'test_collections-1449', 'test_collections-1460', 'test_collections-1471', 'test_collections-1488', 'test_collections-1518', 'test_collections-152', 'test_collections-1539', 'test_collections-1546', 'test_collections-1549', 'test_collections-1553', 'test_collections-1560', 'test_collections-1574', 'test_collections-1588', 'test_collections-1599', 'test_collections-1639', 'test_collections-1646', 'test_collections-166', 'test_collections-1660', 'test_collections-1667', 'test_collections-1674', 'test_collections-1677', 'test_collections-1681', 'test_collections-1688', 'test_collections-169', 'test_collections-17', 'test_collections-1702', 'test_collections-1716', 'test_collections-1727', 'test_collections-1767', 'test_collections-1774', 'test_collections-1795', 'test_collections-180', 'test_collections-1802', 'test_collections-1805', 'test_collections-1809', 'test_collections-1816', 'test_collections-1830', 'test_collections-1833', 'test_collections-1844', 'test_collections-1855', 'test_collections-1895', 'test_collections-1902', 'test_collections-191', 'test_collections-1923', 'test_collections-1930', 'test_collections-1933', 'test_collections-1937', 'test_collections-1944', 'test_collections-1958', 'test_collections-1961', 'test_collections-1972', 'test_collections-1983', 'test_collections-208', 'test_collections-219', 'test_collections-24', 'test_collections-259', 'test_collections-266', 'test_collections-269', 'test_collections-280', 'test_collections-294', 'test_collections-297', 'test_collections-3', 'test_collections-308', 'test_collections-319', 'test_collections-336', 'test_collections-347', 'test_collections-38', 'test_collections-387', 'test_collections-394', 'test_collections-397', 'test_collections-408', 'test_collections-41', 'test_collections-422', 'test_collections-425', 'test_collections-436', 'test_collections-447', 'test_collections-450', 'test_collections-464', 'test_collections-475', 'test_collections-515', 'test_collections-52', 'test_collections-522', 'test_collections-525', 'test_collections-529', 'test_collections-536', 'test_collections-550', 'test_collections-553', 'test_collections-564', 'test_collections-575', 'test_collections-592', 'test_collections-622', 'test_collections-63', 'test_collections-643', 'test_collections-650', 'test_collections-653', 'test_collections-657', 'test_collections-664', 'test_collections-678', 'test_collections-681', 'test_collections-692', 'test_collections-703', 'test_collections-720', 'test_collections-750', 'test_collections-771', 'test_collections-778', 'test_collections-781', 'test_collections-785', 'test_collections-792', 'test_collections-80', 'test_collections-806', 'test_collections-809', 'test_collections-820', 'test_collections-831', 'test_collections-848', 'test_collections-899', 'test_collections-906', 'test_collections-909', 'test_collections-91', 'test_collections-913', 'test_collections-920', 'test_collections-934', 'test_collections-937', 'test_collections-948', 'test_collections-959', 'test_collections-976', 'test_collections-987'], 'INSERT': {}, 'UPDATE': {}}}, 'last': {}, 'a0:2': {}, 'a0': {'UgD.-R.d9qWHq9GKA7rCiaOvTyo': {'DELETE': [], 'INSERT': {u'Basha White': {u'Working_country': u'AUSTRALIA', u'dept': u'Info-Tech', u'salary': 78663, u'is_manager': 0, u'citizen_of': u'USA', u'join_mo': 9, u'join_day': 13, u'skills': [u'Development', u'C++', u'Java'], u'languanges_known': {u'second': u'Thai', u'third': u'English', u'first': u'Dutch'}, u'join_yr': 2012, u'name': u'Basha White', u'email': [u'Basha White-mail@couchbase.com'], u'emp_id': 495}, u'Basha Wilson': {u'Working_country': u'CANADA', u'dept': u'Info-Tech', u'salary': 99199, u'is_manager': 0, u'citizen_of': u'CANADA', u'join_mo': 9, u'join_day': 20, u'skills': [u'Development', u'Testing', u'Java'], u'languanges_known': {u'second': u'Spanish', u'third': u'Quechua', u'first': u'Spanish'}, u'join_yr': 2013, u'name': u'Basha Wilson', u'email': [u'Basha Wilson-mail@couchbase.com'], u'emp_id': 5245}, u'Alvita Jackson': {u'manages': {u'reports': [u'Hedda Davis', u'Riona Davis', u'Hedda Young', u'Adrianne Davis', u'Deirdre Jackson', u'Kallie Nicolas'], u'team_size': 6}, u'Working_country': u'CANADA', u'dept': u'Pre-Sales', u'salary': 192591, u'is_manager': 1, u'citizen_of': u'India', u'join_mo': 9, u'join_day': 19, u'skills': [u'C++', u'Java', u'Python'], u'languanges_known': {u'second': u'Malay', u'third': u'Italian', u'first': u'Dutch'}, u'join_yr': 2011, u'name': u'Alvita Jackson', u'email': [u'Alvita Jackson-mail@couchbase.com'], u'emp_id': 5043}, u'Basha Bailey': {u'manages': {u'reports': [u'Adrianne Russell', u'Ambika Sebastian', u'Basha Wood', u'Hedda Julian'], u'team_size': 4}, u'Working_country': u'AUSTRALIA', u'dept': u'Engineer', u'salary': 70689, u'is_manager': 1, u'citizen_of': u'CANADA', u'join_mo': 9, u'join_day': 4, u'skills': [u'Testing', u'Development', u'C++'], u'languanges_known': {u'second': u'Sinhalese', u'third': u'Quechua', u'first': u'Hindi'}, u'join_yr': 2015, u'name': u'Basha Bailey', u'email': [u'Basha Bailey-mail@couchbase.com'], u'emp_id': 2074}, u'Basha Juan Jose': {u'manages': {u'reports': [u'Cassandra Thomas', u'Hedda Young', u'Fantine Green', u'Lilith Wood', u'Trista Brown', u'Fantine Russell'], u'team_size': 6}, u'Working_country': u'USA', u'dept': u'Support', u'salary': 97368, u'is_manager': 1, u'citizen_of': u'AUSTRALIA', u'join_mo': 9, u'join_day': 9, u'skills': [u'Testing', u'C++', u'Python'], u'languanges_known': {u'second': u'Spanish', u'third': u'Italian', u'first': u'Quechua'}, u'join_yr': 2015, u'name': u'Basha Juan Jose', u'email': [u'Basha Juan Jose-mail@couchbase.com'], u'emp_id': 3144}, u'Alvita Cook': {u'Working_country': u'CANADA', u'dept': u'Support', u'salary': 110274, u'is_manager': 0, u'citizen_of': u'AUSTRALIA', u'join_mo': 9, u'join_day': 3, u'skills': [u'Python', u'C++', u'Testing'], u'languanges_known': {u'second': u'Vietnamese', u'third': u'German', u'first': u'English'}, u'join_yr': 2016, u'name': u'Alvita Cook', u'email': [u'Alvita Cook-mail@couchbase.com'], u'emp_id': 5783}, u'Alvita Baker': {u'Working_country': u'USA', u'dept': u'', u'salary': 104418, u'is_manager': 0, u'citizen_of': u'India', u'join_mo': 9, u'join_day': 2, u'skills': [u'Python', u'C++', u'Testing'], u'languanges_known': {u'second': u'Portuguese', u'third': u'English', u'first': u'Hindi'}, u'join_yr': 2011, u'name': u'Alvita Baker', u'email': [u'Alvita Baker-mail@couchbase.com'], u'emp_id': 5200}, u'Alvita Moore': {u'manages': {u'reports': [u'Cassandra Evans', u'Lilith Wilson', u'Basha Scott', u'Kacila Cook'], u'team_size': 4}, u'Working_country': u'AUSTRALIA', u'dept': u'Accounts', u'salary': 14702, u'is_manager': 1, u'citizen_of': u'AUSTRALIA', u'join_mo': 9, u'join_day': 16, u'skills': [u'Development', u'Testing', u'Java'], u'languanges_known': {u'second': u'Italian', u'third': u'Thai', u'first': u'Nepalese'}, u'join_yr': 2016, u'name': u'Alvita Moore', u'email': [u'Alvita Moore-mail@couchbase.com'], u'emp_id': 9157}, u'Alvita Palmer': {u'manages': {u'reports': [u'Mia Morgan', u'Bryn Carter', u'Adrianne Morgan', u'Drucilla Scott'], u'team_size': 4}, u'Working_country': u'India', u'dept': u'Cloud', u'salary': 69959, u'is_manager': 1, u'citizen_of': u'CANADA', u'join_mo': 9, u'join_day': 14, u'skills': [u'Java', u'Python', u'C++'], u'languanges_known': {u'second': u'Italian', u'third': u'Sinhalese', u'first': u'Spanish'}, u'join_yr': 2015, u'name': u'Alvita Palmer', u'email': [u'Alvita Palmer-mail@couchbase.com'], u'emp_id': 5128}, u'Alvita Howard Jr.': {u'manages': {u'reports': [u'Adrianne Carter', u'Lysha Damian', u'Alvita Baker', u'Deirdre Wood', u'Ambika Young', u'Mia Jackson', u'Riona Julian'], u'team_size': 7}, u'Working_country': u'CANADA', u'dept': u'Accounts', u'salary': 51818, u'is_manager': 1, u'citizen_of': u'CANADA', u'join_mo': 9, u'join_day': 13, u'skills': [u'C++', u'Testing', u'Java'], u'languanges_known': {u'second': u'Japanese', u'third': u'Quechua', u'first': u'Japanese'}, u'join_yr': 2010, u'name': u'Alvita Howard Jr.', u'email': [u'Alvita Howard Jr.-mail@couchbase.com'], u'emp_id': 5194}}, 'UPDATE': {}}}, 'a1': {}}, ['a0', 'a1', 'a0:2', 'a0:3', 'last']]] 2020-10-02 05:30:50,516 | test | INFO | MainThread | [N1qlBase:validate_keys:642] Expected keys: 1841, Actual: 2010, Deleted: 169 FAIL 2020-10-02 05:30:51,207 | test | INFO | MainThread | [collections_base:tearDown:86] Bucket: UgD, Resident ratio(DGM): 100% 2020-10-02 05:30:53,982 | test | INFO | MainThread | [basetestcase:check_coredump_exist:704] 172.23.105.229 : SSH Successful 2020-10-02 05:30:53,982 | test | INFO | MainThread | [basetestcase:check_coredump_exist:705] 172.23.105.229 : Looking for crash dump files 2020-10-02 05:30:54,003 | test | INFO | MainThread | [basetestcase:check_coredump_exist:724] 172.23.105.229 : No crash files found 2020-10-02 05:30:54,003 | test | INFO | MainThread | [basetestcase:check_coredump_exist:726] 172.23.105.229 : Looking for CRITICAL messages in log 2020-10-02 05:30:54,996 | test | INFO | MainThread | [basetestcase:check_coredump_exist:704] 172.23.105.230 : SSH Successful 2020-10-02 05:30:54,996 | test | INFO | MainThread | [basetestcase:check_coredump_exist:705] 172.23.105.230 : Looking for crash dump files 2020-10-02 05:30:55,017 | test | INFO | MainThread | [basetestcase:check_coredump_exist:724] 172.23.105.230 : No crash files found 2020-10-02 05:30:55,017 | test | INFO | MainThread | [basetestcase:check_coredump_exist:726] 172.23.105.230 : Looking for CRITICAL messages in log 2020-10-02 05:30:55,520 | test | INFO | MainThread | [basetestcase:fetch_cb_collect_logs:605] Running cbcollect on node 172.23.105.229 2020-10-02 05:30:55,548 | test | INFO | MainThread | [rest_client:_http_request:728] {'date': 'Fri, 02 Oct 2020 12:30:54 GMT', 'content-length': '2', 'server': 'Couchbase Server', 'expires': 'Thu, 01 Jan 1970 00:00:00 GMT', 'x-frame-options': 'DENY', 'x-permitted-cross-domain-policies': 'none', 'pragma': 'no-cache', 'x-content-type-options': 'nosniff', 'x-xss-protection': '1; mode=block', 'connection': 'close', 'content-type': 'application/json', 'cache-control': 'no-cache,no-store,must-revalidate', 'status': '200'} 2020-10-02 05:30:55,549 | test | INFO | MainThread | [rest_client:_http_request:729] [] 2020-10-02 05:31:05,558 | test | INFO | MainThread | [basetestcase:fetch_cb_collect_logs:609] 172.23.105.229 - cbcollect status: True 2020-10-02 05:31:05,559 | test | INFO | MainThread | [basetestcase:fetch_cb_collect_logs:612] Polling active_tasks to check cbcollect status 2020-10-02 05:33:45,875 | test | INFO | MainThread | [basetestcase:fetch_cb_collect_logs:629] Copying cbcollect ZIP file to Client 2020-10-02 05:33:46,424 | test | INFO | MainThread | [basetestcase:fetch_cb_collect_logs:638] 172.23.105.229 node cb collect zip coped on client : True 2020-10-02 05:33:46,438 | test | INFO | MainThread | [basetestcase:fetch_cb_collect_logs:605] Running cbcollect on node 172.23.105.230 2020-10-02 05:33:46,463 | test | INFO | MainThread | [rest_client:_http_request:728] {'date': 'Fri, 02 Oct 2020 12:33:45 GMT', 'content-length': '2', 'server': 'Couchbase Server', 'expires': 'Thu, 01 Jan 1970 00:00:00 GMT', 'x-frame-options': 'DENY', 'x-permitted-cross-domain-policies': 'none', 'pragma': 'no-cache', 'x-content-type-options': 'nosniff', 'x-xss-protection': '1; mode=block', 'connection': 'close', 'content-type': 'application/json', 'cache-control': 'no-cache,no-store,must-revalidate', 'status': '200'} 2020-10-02 05:33:46,463 | test | INFO | MainThread | [rest_client:_http_request:729] [] 2020-10-02 05:33:56,470 | test | INFO | MainThread | [basetestcase:fetch_cb_collect_logs:609] 172.23.105.230 - cbcollect status: True 2020-10-02 05:33:56,470 | test | INFO | MainThread | [basetestcase:fetch_cb_collect_logs:612] Polling active_tasks to check cbcollect status 2020-10-02 05:35:56,686 | test | INFO | MainThread | [basetestcase:fetch_cb_collect_logs:629] Copying cbcollect ZIP file to Client 2020-10-02 05:35:57,204 | test | INFO | MainThread | [basetestcase:fetch_cb_collect_logs:638] 172.23.105.230 node cb collect zip coped on client : True 2020-10-02 05:35:57,220 | test | CRITICAL | MainThread | [basetestcase:tearDownEverything:490] Skipping get_trace !! 2020-10-02 05:36:09,881 | test | INFO | MainThread | [rest_client:monitorRebalance:1433] Rebalance done. Taken 11.0650000572 seconds to complete 2020-10-02 05:36:09,890 | test | INFO | MainThread | [common_lib:sleep:22] Sleep 5 seconds. Reason: Wait after rebalance complete ====================================================================== FAIL: test_concurrent_txn (N1qlTransaction.basic_ops.BasicOps) ---------------------------------------------------------------------- Traceback (most recent call last): During the test, Remote Connections: 86, Disconnections: 77 SDK Connections: 90, Disconnections: 90 File "pytests/N1qlTransaction/basic_ops.py", line 49, in test_concurrent_txn self.fail("One of the thread failed with unexpected errors") AssertionError: One of the thread failed with unexpected errors ---------------------------------------------------------------------- Ran 1 test in 539.114s FAILED (failures=1) summary so far suite N1qlTransaction.basic_ops.BasicOps , pass 2 , fail 1 failures so far... N1qlTransaction.basic_ops.BasicOps.test_concurrent_txn testrunner logs, diags and results are available under /data/workspace/temp_atomicity_phase1/logs/testrunner-20-Oct-02_05-19-23/test_3