/Users/ashwin/jython2.7.0/bin/jython -Dpython.path=/Users/ashwin/gitRepos/couchbaselabs/TAF/doc_operations_sdk3:/Users/ashwin/gitRepos/couchbaselabs/TAF/doc_operations_sdk3/src/main/java -J-cp jars/sdk3.0/*:jars/* /Users/ashwin/gitRepos/couchbaselabs/TAF/testrunner.py -i node.ini -c debug.conf -m rest -p durability=MAJORITY
Console output is saving to: /Users/ashwin/gitRepos/couchbaselabs/TAF/logs/lastRun.log
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.python.core.PySystemState (file:/Users/ashwin/jython2.7.0/jython.jar) to method java.io.Console.encoding()
WARNING: Please consider reporting this to the maintainers of org.python.core.PySystemState
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
filename: debug.conf
prefix: crash_test.crash_process.CrashTest
Global Test input params:
{'cluster_name': 'node',
 'conf_file': 'debug.conf',
 'durability': 'MAJORITY',
 'ini': 'node.ini',
 'num_nodes': 4,
 'spec': 'debug'}
Logs will be stored at /Users/ashwin/gitRepos/couchbaselabs/TAF/logs/testrunner-19-Jun-18_16-46-32/test_1

./testrunner -i node.ini durability=MAJORITY -t crash_test.crash_process.CrashTest.crash_process,nodes_init=2,replicas=1,num_items=10000,process=memcached,service=memcached,sig_type=sigkill,target_node=replica,GROUP=P0

Test Input params:
{'sig_type': 'sigkill', 'cluster_name': 'node', 'process': 'memcached', 'ini': 'node.ini', 'replicas': '1', 'durability': 'MAJORITY', 'conf_file': 'debug.conf', 'target_node': 'replica', 'spec': 'debug', 'num_nodes': 4, 'logs_folder': '/Users/ashwin/gitRepos/couchbaselabs/TAF/logs/testrunner-19-Jun-18_16-46-32/test_1', 'nodes_init': '2', 'GROUP': 'P0', 'num_items': '10000', 'service': 'memcached', 'case_number': 1}
2019-06-18 16:46:32,448 | root  | WARNING | MainThread | [data_helper:<module>:30] Can not import concurrent module. Data for each server will be loaded/retrieved sequentially
crash_process (crash_test.crash_process.CrashTest) ... 2019-06-18 16:46:33,881 | test  | INFO    | MainThread | [basetestcase:_BaseTestCase__log_setup_status:320] ========= basetestcase setup started for test #1 crash_process =========
2019-06-18 16:46:34,190 | test  | WARNING | MainThread | [basetestcase:setUp:172] any cluster operation in setup will be skipped
2019-06-18 16:46:34,191 | test  | INFO    | MainThread | [basetestcase:_BaseTestCase__log_setup_status:320] ========= basetestcase setup finished for test #1 crash_process =========
2019-06-18 16:46:36,384 | test  | INFO    | pool-2-thread-1 | [table_view:display:72] Rebalance Overview
+----------------+----------+--------------+
| Nodes          | Services | Status       |
+----------------+----------+--------------+
| 10.112.191.101 | [u'kv']  | Cluster node |
| 10.112.191.102 | None     | <--- IN ---  |
+----------------+----------+--------------+

2019-06-18 16:46:36,411 | test  | INFO    | pool-2-thread-1 | [task:check:299] Rebalance - status: running, progress: 0
2019-06-18 16:46:39,457 | test  | INFO    | pool-2-thread-1 | [task:check:299] Rebalance - status: none, progress: 100
2019-06-18 16:46:39,476 | test  | INFO    | pool-2-thread-1 | [task:check:358] Rebalance completed with progress: 100% in 3.08899998665 sec
2019-06-18 16:46:39,510 | test  | INFO    | MainThread | [bucket_ready_functions:create_bucket:99] Creating bucket: default
2019-06-18 16:46:39,568 | infra | INFO    | pool-2-thread-2 | [BucketOperations_Rest:create_bucket:301] http://10.112.191.101:8091/pools/default/buckets with param: replicaIndex=0&maxTTL=0&flushEnabled=1&compressionMode=off&bucketType=membase&name=default&replicaNumber=1&ramQuotaMB=255&threadsNumber=3&evictionPolicy=valueOnly
2019-06-18 16:46:39,591 | infra | INFO    | pool-2-thread-2 | [BucketOperations_Rest:create_bucket:324] 0.02 seconds to create bucket default
2019-06-18 16:46:39,592 | test  | INFO    | MainThread | [bucket_ready_functions:sleep:84] sleep for 2 secs.  ...
2019-06-18 16:46:41,660 | test  | INFO    | MainThread | [bucket_ready_functions:_wait_warmup_completed:2051] Waiting for bucket default to complete warmup
2019-06-18 16:46:43,700 | test  | INFO    | MainThread | [bucket_ready_functions:add_rbac_user:2137] **** add built-in 'cbadminbucket' user to node 10.112.191.101 ****
2019-06-18 16:46:43,811 | test  | INFO    | MainThread | [bucket_ready_functions:add_rbac_user:2141] **** add 'admin' role to 'cbadminbucket' user ****
2019-06-18 16:46:56,246 | infra | INFO    | pool-2-thread-3 | [task:call:1095] Items loaded in task LoadDocumentsTask-_0_1250_create_ are 1250
2019-06-18 16:46:57,052 | infra | INFO    | pool-2-thread-3 | [task:call:1095] Items loaded in task LoadDocumentsTask-_1250_2500_create_ are 1250
2019-06-18 16:46:57,053 | infra | INFO    | pool-2-thread-3 | [task:call:1095] Items loaded in task LoadDocumentsTask-_2500_3750_create_ are 1250
2019-06-18 16:46:57,336 | infra | INFO    | pool-2-thread-3 | [task:call:1095] Items loaded in task LoadDocumentsTask-_3750_5000_create_ are 1250
2019-06-18 16:46:57,338 | infra | INFO    | pool-2-thread-3 | [task:call:1095] Items loaded in task LoadDocumentsTask-_5000_6250_create_ are 1250
2019-06-18 16:46:57,339 | infra | INFO    | pool-2-thread-3 | [task:call:1095] Items loaded in task LoadDocumentsTask-_6250_7500_create_ are 1250
2019-06-18 16:46:57,341 | infra | INFO    | pool-2-thread-3 | [task:call:1095] Items loaded in task LoadDocumentsTask-_7500_8750_create_ are 1250
2019-06-18 16:46:57,342 | infra | INFO    | pool-2-thread-3 | [task:call:1095] Items loaded in task LoadDocumentsTask-_8750_10000_create_ are 1250
2019-06-18 16:46:57,694 | test  | INFO    | pool-2-thread-4 | [table_view:display:72] Ops Trend for bucket 'default'
+----------+----------------------------------------------------------------------------------------------------------+----------+
| Min      | Trend                                                                                                    | Max      |
+----------+----------------------------------------------------------------------------------------------------------+----------+
| 0.000    | **..................................................*..................................................X | 2114.228 |
| 2007.835 | .............................................................................................X           | 2007.835 |
+----------+----------------------------------------------------------------------------------------------------------+----------+

2019-06-18 16:46:57,697 | infra | WARNING | pool-2-thread-3 | [task_manager:print_tasks_in_pool:70] Task 'DocumentsLoadGenTask_1560856611.49' not completed
2019-06-18 16:46:57,697 | infra | INFO    | pool-2-thread-3 | [task:call:1128] Task 'LoadDocumentsTask-_0_1250_create_' complete. Loaded 1250 items
2019-06-18 16:46:57,698 | infra | INFO    | pool-2-thread-3 | [task:call:1128] Task 'LoadDocumentsTask-_1250_2500_create_' complete. Loaded 1250 items
2019-06-18 16:46:57,700 | infra | INFO    | pool-2-thread-3 | [task:call:1128] Task 'LoadDocumentsTask-_2500_3750_create_' complete. Loaded 1250 items
2019-06-18 16:46:57,700 | infra | INFO    | pool-2-thread-3 | [task:call:1128] Task 'LoadDocumentsTask-_3750_5000_create_' complete. Loaded 1250 items
2019-06-18 16:46:57,700 | infra | INFO    | pool-2-thread-3 | [task:call:1128] Task 'LoadDocumentsTask-_5000_6250_create_' complete. Loaded 1250 items
2019-06-18 16:46:57,700 | infra | INFO    | pool-2-thread-3 | [task:call:1128] Task 'LoadDocumentsTask-_6250_7500_create_' complete. Loaded 1250 items
2019-06-18 16:46:57,701 | infra | INFO    | pool-2-thread-3 | [task:call:1128] Task 'LoadDocumentsTask-_7500_8750_create_' complete. Loaded 1250 items
2019-06-18 16:46:57,701 | infra | INFO    | pool-2-thread-3 | [task:call:1128] Task 'LoadDocumentsTask-_8750_10000_create_' complete. Loaded 1250 items
2019-06-18 16:46:58,546 | test  | WARNING | pool-2-thread-7 | [task:_get_all_stats_and_compare:1564] Not Ready: ep_queue_size 1539 == 0. Received: {'10.112.191.101': '1539'} for bucket 'default'
2019-06-18 16:46:58,957 | test  | WARNING | pool-2-thread-1 | [task:_get_all_stats_and_compare:1564] Not Ready: ep_queue_size 322 == 0. Received: {'10.112.191.102': '322'} for bucket 'default'
2019-06-18 16:47:04,785 | test  | INFO    | MainThread | [bucket_ready_functions:sleep:84] sleep for 5 secs.  ...
2019-06-18 16:47:09,973 | test  | INFO    | MainThread | [crash_process:setUp:54] ==========Finished CrashTest setup========
2019-06-18 16:47:15,601 | test  | INFO    | MainThread | [basetestcase:sleep:372] Wait for doc_ops to start. Sleep for 10 secs ...
2019-06-18 16:47:25,611 | test  | INFO    | MainThread | [crash_process:crash_process:164] Killing memcached:memcached on node 10.112.191.101

Process finished with exit code 137 (interrupted by signal 9: SIGKILL)