filename: ./4node.conf Global Test input params: {'cluster_name': '4node', 'conf_file': './4node.conf', 'ini': './4node.ini', 'num_nodes': 4, 'spec': '4node'} Logs will be stored at /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_1 ./testrunner -i ./4node.ini -t recreatebuckettests.RecreateMembaseBuckets.test_default_moxi Test Input params: {'conf_file': './4node.conf', 'num_nodes': 4, 'cluster_name': '4node', 'ini': './4node.ini', 'case_number': 1, 'logs_folder': '/testrunner/logs/testrunner-18-Jan-25_21-16-28/test_1', 'spec': '4node'} Run before suite setup for recreatebuckettests.RecreateMembaseBuckets.test_default_moxi 2018-01-25 21:16:28 | WARNING | MainProcess | MainThread | [data_helper.] Can not import concurrent module. Data for each server will be loaded/retrieved sequentially 2018-01-25 21:16:28 | INFO | MainProcess | test_thread | [bucket_helper.delete_all_buckets_or_assert] deleting existing buckets [] on 172.17.0.8 2018-01-25 21:16:28 | INFO | MainProcess | test_thread | [bucket_helper.delete_all_buckets_or_assert] sleep 2 seconds to make sure all buckets were deleted completely. 2018-01-25 21:16:30 | INFO | MainProcess | test_thread | [bucket_helper.delete_all_buckets_or_assert] deleting existing buckets [] on 172.17.0.6 2018-01-25 21:16:30 | INFO | MainProcess | test_thread | [bucket_helper.delete_all_buckets_or_assert] sleep 2 seconds to make sure all buckets were deleted completely. 2018-01-25 21:16:32 | INFO | MainProcess | test_thread | [bucket_helper.delete_all_buckets_or_assert] deleting existing buckets [] on 172.17.0.7 2018-01-25 21:16:32 | INFO | MainProcess | test_thread | [bucket_helper.delete_all_buckets_or_assert] sleep 2 seconds to make sure all buckets were deleted completely. 2018-01-25 21:16:34 | INFO | MainProcess | test_thread | [bucket_helper.delete_all_buckets_or_assert] deleting existing buckets [] on 172.17.0.5 2018-01-25 21:16:34 | INFO | MainProcess | test_thread | [bucket_helper.delete_all_buckets_or_assert] sleep 2 seconds to make sure all buckets were deleted completely. 2018-01-25 21:16:36 | ERROR | MainProcess | test_thread | [rest_client._http_request] DELETE http://172.17.0.8:8091/settings/rbac/users/local/cbadminbucket body: headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 404 reason: unknown "User was not found." auth: Administrator:password 2018-01-25 21:16:36 | INFO | MainProcess | test_thread | [internal_user.delete_user] Exception while deleting user. Exception is -"User was not found." 2018-01-25 21:16:56 | INFO | MainProcess | test_thread | [rest_client.get_nodes_versions] Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] 2018-01-25 21:16:56 | INFO | MainProcess | test_thread | [rest_client.create_bucket] http://172.17.0.8:8091/pools/default/buckets with param: bucketType=membase&threadsNumber=3&authType=sasl&replicaIndex=1&name=default&saslPassword=&evictionPolicy=valueOnly&flushEnabled=1&replicaNumber=1&ramQuotaMB=200 2018-01-25 21:16:56 | INFO | MainProcess | test_thread | [rest_client.create_bucket] 0.01 seconds to create bucket default 2018-01-25 21:16:56 | INFO | MainProcess | test_thread | [bucket_helper.wait_for_bucket_creation] waiting for bucket creation to complete.... 2018-01-25 21:16:56 | INFO | MainProcess | test_thread | [rest_client.bucket_exists] node 172.17.0.8 existing buckets : [u'default'] 2018-01-25 21:16:56 | INFO | MainProcess | test_thread | [rest_client.bucket_exists] node 172.17.0.8 found bucket default 2018-01-25 21:16:56 | INFO | MainProcess | test_thread | [bucket_helper.wait_for_memcached] waiting for memcached bucket : default in 172.17.0.8 to accept set ops 2018-01-25 21:16:57 | INFO | MainProcess | test_thread | [data_helper.direct_client] creating direct client cb-example-0000.cb-example.default.svc:11210 default 2018-01-25 21:16:58 | INFO | MainProcess | test_thread | [data_helper.direct_client] creating direct client cb-example-0001.cb-example.default.svc:11210 default 2018-01-25 21:16:58 | INFO | MainProcess | test_thread | [data_helper.direct_client] creating direct client cb-example-0002.cb-example.default.svc:11210 default 2018-01-25 21:16:59 | INFO | MainProcess | test_thread | [data_helper.direct_client] creating direct client cb-example-0003.cb-example.default.svc:11210 default 2018-01-25 21:16:59 | INFO | MainProcess | test_thread | [data_helper.direct_client] creating direct client cb-example-0000.cb-example.default.svc:11210 default 2018-01-25 21:17:00 | INFO | MainProcess | test_thread | [data_helper.direct_client] creating direct client cb-example-0001.cb-example.default.svc:11210 default 2018-01-25 21:17:00 | INFO | MainProcess | test_thread | [data_helper.direct_client] creating direct client cb-example-0002.cb-example.default.svc:11210 default 2018-01-25 21:17:00 | INFO | MainProcess | test_thread | [data_helper.direct_client] creating direct client cb-example-0003.cb-example.default.svc:11210 default 2018-01-25 21:17:00 | INFO | MainProcess | test_thread | [data_helper.direct_client] creating direct client 172.17.0.8:11210 default 2018-01-25 21:17:00 | INFO | MainProcess | test_thread | [bucket_helper.load_some_data] emptySpace : 791359064 fill_ram_percentage : 1 2018-01-25 21:17:00 | INFO | MainProcess | test_thread | [bucket_helper.load_some_data] fill_space 7913590.64 2018-01-25 21:17:00 | INFO | MainProcess | test_thread | [bucket_helper.load_some_data] packetSize: 10240 2018-01-25 21:17:00 | INFO | MainProcess | test_thread | [bucket_helper.load_some_data] memory usage before key insertion : 47501736 2018-01-25 21:17:00 | INFO | MainProcess | test_thread | [bucket_helper.load_some_data] inserting 772 new keys to memcached @ 772 2018-01-25 21:17:00 | ERROR | MainProcess | test_thread | [bucket_helper.load_some_data] Memcached error #7 'Not my vbucket': {"rev":55,"name":"default","uri":"/pools/default/buckets/default?bucket_uuid=f8b6e54e42ee9f286fdc9b2e2ac27d36","streamingUri":"/pools/default/bucketsStreaming/default?bucket_uuid=f8b6e54e42ee9f286fdc9b2e2ac27d36","nodes":[{"couchApiBase":"http://cb-example-0000.cb-example.default.svc:8092/default%2Bf8b6e54e42ee9f286fdc9b2e2ac27d36","hostname":"cb-example-0000.cb-example.default.svc:8091","ports":{"proxy":11211,"direct":11210}},{"couchApiBase":"http://cb-example-0001.cb-example.default.svc:8092/default%2Bf8b6e54e42ee9f286fdc9b2e2ac27d36","hostname":"cb-example-0001.cb-example.default.svc:8091","ports":{"proxy":11211,"direct":11210}},{"couchApiBase":"http://cb-example-0002.cb-example.default.svc:8092/default%2Bf8b6e54e42ee9f286fdc9b2e2ac27d36","hostname":"cb-example-0002.cb-example.default.svc:8091","ports":{"proxy":11211,"direct":11210}},{"couchApiBase":"http://cb-example-0003.cb-example.default.svc:8092/default%2Bf8b6e54e42ee9f286fdc9b2e2ac27d36","hostname":"cb-example-0003.cb-example.default.svc:8091","ports":{"proxy":11211,"direct":11210}}],"nodesExt":[{"services":{"mgmt":8091,"mgmtSSL":18091,"fts":8094,"ftsSSL":18094,"indexAdmin":9100,"indexScan":9101,"indexHttp":9102,"indexStreamInit":9103,"indexStreamCatchup":9104,"indexStreamMaint":9105,"indexHttps":19102,"capiSSL":18092,"capi":8092,"kvSSL":11207,"projector":9999,"kv":11210,"moxi":11211,"n1ql":8093,"n1qlSSL":18093},"hostname":"cb-example-0000.cb-example.default.svc"},{"services":{"mgmt":8091,"mgmtSSL":18091,"fts":8094,"ftsSSL":18094,"indexAdmin":9100,"indexScan":9101,"indexHttp":9102,"indexStreamInit":9103,"indexStreamCatchup":9104,"indexStreamMaint":9105,"indexHttps":19102,"capiSSL":18092,"capi":8092,"kvSSL":11207,"projector":9999,"kv":11210,"moxi":11211,"n1ql":8093,"n1qlSSL":18093},"hostname":"cb-example-0001.cb-example.default.svc"},{"services":{"mgmt":8091,"mgmtSSL":18091,"fts":8094,"ftsSSL":18094,"indexAdmin":9100,"indexScan":9101,"indexHttp":9102,"indexStreamInit":9103,"indexStreamCatchup":9104,"indexStreamMaint":9105,"indexHttps":19102,"capiSSL":18092,"capi":8092,"kvSSL":11207,"projector":9999,"kv":11210,"moxi":11211,"n1ql":8093,"n1qlSSL":18093},"hostname":"cb-example-0002.cb-example.default.svc"},{"services":{"mgmt":8091,"mgmtSSL":18091,"fts":8094,"ftsSSL":18094,"indexAdmin":9100,"indexScan":9101,"indexHttp":9102,"indexStreamInit":9103,"indexStreamCatchup":9104,"indexStreamMaint":9105,"indexHttps":19102,"capiSSL":18092,"capi":8092,"kvSSL":11207,"projector":9999,"kv":11210,"moxi":11211,"n1ql":8093,"n1qlSSL":18093},"thisNode":true,"hostname":"cb-example-0003.cb-example.default.svc"}],"nodeLocator":"vbucket","uuid":"f8b6e54e42ee9f286fdc9b2e2ac27d36","ddocs":{"uri":"/pools/default/buckets/default/ddocs"},"vBucketServerMap":{"hashAlgorithm":"CRC","numReplicas":1,"serverList":["cb-example-0000.cb-example.default.svc:11210","cb-example-0001.cb-example.default.svc:11210","cb-example-0002.cb-example.default.svc:11210","cb-example-0003.cb-example.default.svc:11210"],"vBucketMap":[[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,1],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,2],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[0,3],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,0],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,2],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[1,3],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,0],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,1],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[2,3],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,0],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,1],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2],[3,2]]},"bucketCapabilitiesVer":"","bucketCapabilities":["xattr","dcp","cbhello","touch","couchapi","cccp","xdcrCheckpointing","nodesExt"]} for vbucket :253 to mc 172.17.0.8:11210 2018-01-25 21:17:00 | ERROR | MainProcess | test_thread | [bucket_helper.load_some_data] unable to push key : key_903cc765-9561-4e84-a99d-c5cd51bcf721_0 to vbucket : 253 test_default_moxi (recreatebuckettests.RecreateMembaseBuckets) ... ERROR 2018-01-25 21:17:01 | INFO | MainProcess | test_thread | [bucket_helper.delete_all_buckets_or_assert] deleting existing buckets [u'default'] on 172.17.0.8 2018-01-25 21:17:01 | INFO | MainProcess | test_thread | [bucket_helper.delete_all_buckets_or_assert] remove bucket default ... 2018-01-25 21:17:01 | INFO | MainProcess | test_thread | [bucket_helper.delete_all_buckets_or_assert] deleted bucket : default from 172.17.0.8 2018-01-25 21:17:01 | INFO | MainProcess | test_thread | [bucket_helper.wait_for_bucket_deletion] waiting for bucket deletion to complete.... 2018-01-25 21:17:01 | INFO | MainProcess | test_thread | [rest_client.bucket_exists] node 172.17.0.8 existing buckets : [] 2018-01-25 21:17:01 | INFO | MainProcess | test_thread | [bucket_helper.delete_all_buckets_or_assert] sleep 2 seconds to make sure all buckets were deleted completely. 2018-01-25 21:17:03 | INFO | MainProcess | test_thread | [bucket_helper.delete_all_buckets_or_assert] deleting existing buckets [] on 172.17.0.6 2018-01-25 21:17:03 | INFO | MainProcess | test_thread | [bucket_helper.delete_all_buckets_or_assert] sleep 2 seconds to make sure all buckets were deleted completely. 2018-01-25 21:17:05 | INFO | MainProcess | test_thread | [bucket_helper.delete_all_buckets_or_assert] deleting existing buckets [] on 172.17.0.7 2018-01-25 21:17:05 | INFO | MainProcess | test_thread | [bucket_helper.delete_all_buckets_or_assert] sleep 2 seconds to make sure all buckets were deleted completely. 2018-01-25 21:17:07 | INFO | MainProcess | test_thread | [bucket_helper.delete_all_buckets_or_assert] deleting existing buckets [] on 172.17.0.5 2018-01-25 21:17:07 | INFO | MainProcess | test_thread | [bucket_helper.delete_all_buckets_or_assert] sleep 2 seconds to make sure all buckets were deleted completely. ====================================================================== ERROR: test_default_moxi (recreatebuckettests.RecreateMembaseBuckets) ---------------------------------------------------------------------- Traceback (most recent call last): File "pytests/recreatebuckettests.py", line 78, in test_default_moxi inserted_keys = BucketOperationHelper.load_some_data(serverInfo, 1, name) File "lib/membase/helper/bucket_helper.py", line 524, in load_some_data if test: NameError: global name 'test' is not defined ---------------------------------------------------------------------- Ran 1 test in 41.833s FAILED (errors=1) summary so far suite recreatebuckettests.RecreateMembaseBuckets , pass 0 , fail 1 failures so far... recreatebuckettests.RecreateMembaseBuckets.test_default_moxi testrunner logs, diags and results are available under /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_1 Logs will be stored at /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_2 ./testrunner -i ./4node.ini -t deletebuckettests.DeleteMembaseBuckets.test_non_default_moxi Test Input params: {'conf_file': './4node.conf', 'num_nodes': 4, 'cluster_name': '4node', 'ini': './4node.ini', 'case_number': 2, 'logs_folder': '/testrunner/logs/testrunner-18-Jan-25_21-16-28/test_2', 'spec': '4node'} [2018-01-25 21:17:10,049] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.8 [2018-01-25 21:17:10,051] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:17:12,061] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.6 [2018-01-25 21:17:12,061] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:17:14,071] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.7 [2018-01-25 21:17:14,071] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:17:16,080] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.5 [2018-01-25 21:17:16,080] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:17:18,102] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:17:18,102] - [rest_client:2237] INFO - http://172.17.0.8:8091/pools/default/buckets with param: bucketType=membase&threadsNumber=3&authType=none&replicaIndex=1&name=new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2&evictionPolicy=valueOnly&flushEnabled=1&replicaNumber=0&ramQuotaMB=200 [2018-01-25 21:17:18,108] - [rest_client:2262] INFO - 0.01 seconds to create bucket new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2 [2018-01-25 21:17:18,109] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 21:17:18,359] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 21:17:18,786] - [bucket_helper:251] INFO - waiting for bucket creation to complete.... [2018-01-25 21:17:18,795] - [rest_client:133] INFO - node 172.17.0.8 existing buckets : [u'new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2'] [2018-01-25 21:17:18,795] - [rest_client:137] INFO - node 172.17.0.8 found bucket new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2 [2018-01-25 21:17:19,302] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:17:19,306] - [rest_client:133] INFO - node 172.17.0.8 existing buckets : [] [2018-01-25 21:17:19,306] - [deletebuckettests:99] INFO - waiting for bucket data files deletion from the disk .... [2018-01-25 21:17:19,318] - [bucket_helper:174] INFO - deleting existing bucket new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2 on ip:172.17.0.8 port:8091 ssh_username:root [2018-01-25 21:17:19,325] - [rest_client:133] INFO - node 172.17.0.8 existing buckets : [] [2018-01-25 21:17:19,325] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:17:19,327] - [rest_client:133] INFO - node 172.17.0.8 existing buckets : [] [2018-01-25 21:17:19,338] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:17:19,338] - [rest_client:2237] INFO - http://172.17.0.8:8091/pools/default/buckets with param: bucketType=membase&threadsNumber=3&authType=none&replicaIndex=1&name=new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2&evictionPolicy=valueOnly&flushEnabled=1&replicaNumber=1&ramQuotaMB=200 [2018-01-25 21:17:19,345] - [rest_client:2262] INFO - 0.01 seconds to create bucket new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2 [2018-01-25 21:17:19,346] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 21:17:19,515] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 21:17:19,985] - [bucket_helper:251] INFO - waiting for bucket creation to complete.... [2018-01-25 21:17:19,991] - [rest_client:133] INFO - node 172.17.0.8 existing buckets : [u'new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2'] [2018-01-25 21:17:19,991] - [rest_client:137] INFO - node 172.17.0.8 found bucket new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2 [2018-01-25 21:17:20,692] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:17:20,695] - [rest_client:133] INFO - node 172.17.0.8 existing buckets : [] [2018-01-25 21:17:20,695] - [deletebuckettests:99] INFO - waiting for bucket data files deletion from the disk .... [2018-01-25 21:17:20,713] - [bucket_helper:174] INFO - deleting existing bucket new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2 on ip:172.17.0.8 port:8091 ssh_username:root [2018-01-25 21:17:20,722] - [rest_client:133] INFO - node 172.17.0.8 existing buckets : [] [2018-01-25 21:17:20,723] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:17:20,726] - [rest_client:133] INFO - node 172.17.0.8 existing buckets : [] [2018-01-25 21:17:20,742] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:17:20,742] - [rest_client:2237] INFO - http://172.17.0.8:8091/pools/default/buckets with param: bucketType=membase&threadsNumber=3&authType=none&replicaIndex=1&name=new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2&evictionPolicy=valueOnly&flushEnabled=1&replicaNumber=2&ramQuotaMB=200 [2018-01-25 21:17:20,752] - [rest_client:2262] INFO - 0.01 seconds to create bucket new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2 [2018-01-25 21:17:20,759] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 21:17:21,058] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 21:17:21,499] - [bucket_helper:251] INFO - waiting for bucket creation to complete.... [2018-01-25 21:17:21,507] - [rest_client:133] INFO - node 172.17.0.8 existing buckets : [u'new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2'] [2018-01-25 21:17:21,507] - [rest_client:137] INFO - node 172.17.0.8 found bucket new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2 [2018-01-25 21:17:22,083] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:17:22,086] - [rest_client:133] INFO - node 172.17.0.8 existing buckets : [] [2018-01-25 21:17:22,086] - [deletebuckettests:99] INFO - waiting for bucket data files deletion from the disk .... [2018-01-25 21:17:22,097] - [bucket_helper:174] INFO - deleting existing bucket new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2 on ip:172.17.0.8 port:8091 ssh_username:root [2018-01-25 21:17:22,105] - [rest_client:133] INFO - node 172.17.0.8 existing buckets : [] [2018-01-25 21:17:22,105] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:17:22,107] - [rest_client:133] INFO - node 172.17.0.8 existing buckets : [] [2018-01-25 21:17:22,119] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:17:22,119] - [rest_client:2237] INFO - http://172.17.0.8:8091/pools/default/buckets with param: bucketType=membase&threadsNumber=3&authType=none&replicaIndex=1&name=new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2&evictionPolicy=valueOnly&flushEnabled=1&replicaNumber=3&ramQuotaMB=200 [2018-01-25 21:17:22,126] - [rest_client:2262] INFO - 0.01 seconds to create bucket new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2 [2018-01-25 21:17:22,127] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 21:17:22,323] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 21:17:22,715] - [bucket_helper:251] INFO - waiting for bucket creation to complete.... [2018-01-25 21:17:22,722] - [rest_client:133] INFO - node 172.17.0.8 existing buckets : [u'new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2'] [2018-01-25 21:17:22,722] - [rest_client:137] INFO - node 172.17.0.8 found bucket new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2 [2018-01-25 21:17:23,325] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:17:23,328] - [rest_client:133] INFO - node 172.17.0.8 existing buckets : [] [2018-01-25 21:17:23,328] - [deletebuckettests:99] INFO - waiting for bucket data files deletion from the disk .... [2018-01-25 21:17:23,344] - [bucket_helper:174] INFO - deleting existing bucket new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2 on ip:172.17.0.8 port:8091 ssh_username:root [2018-01-25 21:17:23,356] - [rest_client:133] INFO - node 172.17.0.8 existing buckets : [] [2018-01-25 21:17:23,356] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:17:23,358] - [rest_client:133] INFO - node 172.17.0.8 existing buckets : [] [2018-01-25 21:17:23,373] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:17:23,373] - [rest_client:2237] INFO - http://172.17.0.6:8091/pools/default/buckets with param: bucketType=membase&threadsNumber=3&authType=none&replicaIndex=1&name=new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2&evictionPolicy=valueOnly&flushEnabled=1&replicaNumber=0&ramQuotaMB=200 [2018-01-25 21:17:23,401] - [rest_client:2262] INFO - 0.03 seconds to create bucket new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2 [2018-01-25 21:17:23,404] - [remote_util:232] INFO - connecting to 172.17.0.6 with username:root password:couchbase ssh_key: [2018-01-25 21:17:23,714] - [remote_util:266] INFO - Connected to 172.17.0.6 [2018-01-25 21:17:24,338] - [bucket_helper:251] INFO - waiting for bucket creation to complete.... [2018-01-25 21:17:24,344] - [rest_client:133] INFO - node 172.17.0.6 existing buckets : [u'new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2'] [2018-01-25 21:17:24,344] - [rest_client:137] INFO - node 172.17.0.6 found bucket new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2 [2018-01-25 21:17:24,824] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:17:24,827] - [rest_client:133] INFO - node 172.17.0.6 existing buckets : [] [2018-01-25 21:17:24,827] - [deletebuckettests:99] INFO - waiting for bucket data files deletion from the disk .... [2018-01-25 21:17:24,839] - [bucket_helper:174] INFO - deleting existing bucket new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2 on ip:172.17.0.6 port:8091 ssh_username:root [2018-01-25 21:17:24,845] - [rest_client:133] INFO - node 172.17.0.6 existing buckets : [] [2018-01-25 21:17:24,845] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:17:24,848] - [rest_client:133] INFO - node 172.17.0.6 existing buckets : [] [2018-01-25 21:17:24,858] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:17:24,858] - [rest_client:2237] INFO - http://172.17.0.6:8091/pools/default/buckets with param: bucketType=membase&threadsNumber=3&authType=none&replicaIndex=1&name=new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2&evictionPolicy=valueOnly&flushEnabled=1&replicaNumber=1&ramQuotaMB=200 [2018-01-25 21:17:24,867] - [rest_client:2262] INFO - 0.01 seconds to create bucket new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2 [2018-01-25 21:17:24,868] - [remote_util:232] INFO - connecting to 172.17.0.6 with username:root password:couchbase ssh_key: [2018-01-25 21:17:25,074] - [remote_util:266] INFO - Connected to 172.17.0.6 [2018-01-25 21:17:25,457] - [bucket_helper:251] INFO - waiting for bucket creation to complete.... [2018-01-25 21:17:25,464] - [rest_client:133] INFO - node 172.17.0.6 existing buckets : [u'new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2'] [2018-01-25 21:17:25,464] - [rest_client:137] INFO - node 172.17.0.6 found bucket new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2 [2018-01-25 21:17:25,925] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:17:25,928] - [rest_client:133] INFO - node 172.17.0.6 existing buckets : [] [2018-01-25 21:17:25,928] - [deletebuckettests:99] INFO - waiting for bucket data files deletion from the disk .... [2018-01-25 21:17:25,938] - [bucket_helper:174] INFO - deleting existing bucket new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2 on ip:172.17.0.6 port:8091 ssh_username:root [2018-01-25 21:17:25,943] - [rest_client:133] INFO - node 172.17.0.6 existing buckets : [] [2018-01-25 21:17:25,943] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:17:25,945] - [rest_client:133] INFO - node 172.17.0.6 existing buckets : [] [2018-01-25 21:17:25,954] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:17:25,954] - [rest_client:2237] INFO - http://172.17.0.6:8091/pools/default/buckets with param: bucketType=membase&threadsNumber=3&authType=none&replicaIndex=1&name=new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2&evictionPolicy=valueOnly&flushEnabled=1&replicaNumber=2&ramQuotaMB=200 [2018-01-25 21:17:25,961] - [rest_client:2262] INFO - 0.01 seconds to create bucket new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2 [2018-01-25 21:17:25,962] - [remote_util:232] INFO - connecting to 172.17.0.6 with username:root password:couchbase ssh_key: [2018-01-25 21:17:26,204] - [remote_util:266] INFO - Connected to 172.17.0.6 [2018-01-25 21:17:26,530] - [bucket_helper:251] INFO - waiting for bucket creation to complete.... [2018-01-25 21:17:26,535] - [rest_client:133] INFO - node 172.17.0.6 existing buckets : [u'new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2'] [2018-01-25 21:17:26,536] - [rest_client:137] INFO - node 172.17.0.6 found bucket new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2 [2018-01-25 21:17:27,459] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:17:27,462] - [rest_client:133] INFO - node 172.17.0.6 existing buckets : [] [2018-01-25 21:17:27,462] - [deletebuckettests:99] INFO - waiting for bucket data files deletion from the disk .... [2018-01-25 21:17:27,479] - [bucket_helper:174] INFO - deleting existing bucket new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2 on ip:172.17.0.6 port:8091 ssh_username:root [2018-01-25 21:17:27,485] - [rest_client:133] INFO - node 172.17.0.6 existing buckets : [] [2018-01-25 21:17:27,485] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:17:27,487] - [rest_client:133] INFO - node 172.17.0.6 existing buckets : [] [2018-01-25 21:17:27,497] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:17:27,498] - [rest_client:2237] INFO - http://172.17.0.6:8091/pools/default/buckets with param: bucketType=membase&threadsNumber=3&authType=none&replicaIndex=1&name=new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2&evictionPolicy=valueOnly&flushEnabled=1&replicaNumber=3&ramQuotaMB=200 [2018-01-25 21:17:27,504] - [rest_client:2262] INFO - 0.01 seconds to create bucket new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2 [2018-01-25 21:17:27,512] - [remote_util:232] INFO - connecting to 172.17.0.6 with username:root password:couchbase ssh_key: [2018-01-25 21:17:27,745] - [remote_util:266] INFO - Connected to 172.17.0.6 [2018-01-25 21:17:28,120] - [bucket_helper:251] INFO - waiting for bucket creation to complete.... [2018-01-25 21:17:28,130] - [rest_client:133] INFO - node 172.17.0.6 existing buckets : [u'new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2'] [2018-01-25 21:17:28,131] - [rest_client:137] INFO - node 172.17.0.6 found bucket new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2 [2018-01-25 21:17:28,686] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:17:28,689] - [rest_client:133] INFO - node 172.17.0.6 existing buckets : [] [2018-01-25 21:17:28,689] - [deletebuckettests:99] INFO - waiting for bucket data files deletion from the disk .... [2018-01-25 21:17:28,700] - [bucket_helper:174] INFO - deleting existing bucket new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2 on ip:172.17.0.6 port:8091 ssh_username:root [2018-01-25 21:17:28,706] - [rest_client:133] INFO - node 172.17.0.6 existing buckets : [] [2018-01-25 21:17:28,706] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:17:28,709] - [rest_client:133] INFO - node 172.17.0.6 existing buckets : [] [2018-01-25 21:17:28,723] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:17:28,723] - [rest_client:2237] INFO - http://172.17.0.7:8091/pools/default/buckets with param: bucketType=membase&threadsNumber=3&authType=none&replicaIndex=1&name=new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2&evictionPolicy=valueOnly&flushEnabled=1&replicaNumber=0&ramQuotaMB=200 [2018-01-25 21:17:28,738] - [rest_client:2262] INFO - 0.01 seconds to create bucket new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2 [2018-01-25 21:17:28,739] - [remote_util:232] INFO - connecting to 172.17.0.7 with username:root password:couchbase ssh_key: [2018-01-25 21:17:28,977] - [remote_util:266] INFO - Connected to 172.17.0.7 [2018-01-25 21:17:29,365] - [bucket_helper:251] INFO - waiting for bucket creation to complete.... [2018-01-25 21:17:29,371] - [rest_client:133] INFO - node 172.17.0.7 existing buckets : [u'new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2'] [2018-01-25 21:17:29,371] - [rest_client:137] INFO - node 172.17.0.7 found bucket new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2 [2018-01-25 21:17:29,915] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:17:29,917] - [rest_client:133] INFO - node 172.17.0.7 existing buckets : [] [2018-01-25 21:17:29,918] - [deletebuckettests:99] INFO - waiting for bucket data files deletion from the disk .... [2018-01-25 21:17:29,934] - [bucket_helper:174] INFO - deleting existing bucket new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2 on ip:172.17.0.7 port:8091 ssh_username:root [2018-01-25 21:17:29,941] - [rest_client:133] INFO - node 172.17.0.7 existing buckets : [] [2018-01-25 21:17:29,941] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:17:29,943] - [rest_client:133] INFO - node 172.17.0.7 existing buckets : [] [2018-01-25 21:17:29,953] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:17:29,953] - [rest_client:2237] INFO - http://172.17.0.7:8091/pools/default/buckets with param: bucketType=membase&threadsNumber=3&authType=none&replicaIndex=1&name=new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2&evictionPolicy=valueOnly&flushEnabled=1&replicaNumber=1&ramQuotaMB=200 [2018-01-25 21:17:29,962] - [rest_client:2262] INFO - 0.01 seconds to create bucket new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2 [2018-01-25 21:17:29,969] - [remote_util:232] INFO - connecting to 172.17.0.7 with username:root password:couchbase ssh_key: [2018-01-25 21:17:30,403] - [remote_util:266] INFO - Connected to 172.17.0.7 [2018-01-25 21:17:30,782] - [bucket_helper:251] INFO - waiting for bucket creation to complete.... [2018-01-25 21:17:30,788] - [rest_client:133] INFO - node 172.17.0.7 existing buckets : [u'new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2'] [2018-01-25 21:17:30,788] - [rest_client:137] INFO - node 172.17.0.7 found bucket new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2 [2018-01-25 21:17:31,365] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:17:31,368] - [rest_client:133] INFO - node 172.17.0.7 existing buckets : [] [2018-01-25 21:17:31,368] - [deletebuckettests:99] INFO - waiting for bucket data files deletion from the disk .... [2018-01-25 21:17:31,379] - [bucket_helper:174] INFO - deleting existing bucket new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2 on ip:172.17.0.7 port:8091 ssh_username:root [2018-01-25 21:17:31,386] - [rest_client:133] INFO - node 172.17.0.7 existing buckets : [] [2018-01-25 21:17:31,386] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:17:31,388] - [rest_client:133] INFO - node 172.17.0.7 existing buckets : [] [2018-01-25 21:17:31,401] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:17:31,401] - [rest_client:2237] INFO - http://172.17.0.7:8091/pools/default/buckets with param: bucketType=membase&threadsNumber=3&authType=none&replicaIndex=1&name=new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2&evictionPolicy=valueOnly&flushEnabled=1&replicaNumber=2&ramQuotaMB=200 [2018-01-25 21:17:31,408] - [rest_client:2262] INFO - 0.01 seconds to create bucket new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2 [2018-01-25 21:17:31,412] - [remote_util:232] INFO - connecting to 172.17.0.7 with username:root password:couchbase ssh_key: [2018-01-25 21:17:31,623] - [remote_util:266] INFO - Connected to 172.17.0.7 [2018-01-25 21:17:32,004] - [bucket_helper:251] INFO - waiting for bucket creation to complete.... [2018-01-25 21:17:32,011] - [rest_client:133] INFO - node 172.17.0.7 existing buckets : [u'new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2'] [2018-01-25 21:17:32,011] - [rest_client:137] INFO - node 172.17.0.7 found bucket new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2 [2018-01-25 21:17:32,523] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:17:32,533] - [rest_client:133] INFO - node 172.17.0.7 existing buckets : [] [2018-01-25 21:17:32,533] - [deletebuckettests:99] INFO - waiting for bucket data files deletion from the disk .... [2018-01-25 21:17:32,548] - [bucket_helper:174] INFO - deleting existing bucket new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2 on ip:172.17.0.7 port:8091 ssh_username:root [2018-01-25 21:17:32,554] - [rest_client:133] INFO - node 172.17.0.7 existing buckets : [] [2018-01-25 21:17:32,554] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:17:32,556] - [rest_client:133] INFO - node 172.17.0.7 existing buckets : [] [2018-01-25 21:17:32,567] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:17:32,567] - [rest_client:2237] INFO - http://172.17.0.7:8091/pools/default/buckets with param: bucketType=membase&threadsNumber=3&authType=none&replicaIndex=1&name=new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2&evictionPolicy=valueOnly&flushEnabled=1&replicaNumber=3&ramQuotaMB=200 [2018-01-25 21:17:32,576] - [rest_client:2262] INFO - 0.01 seconds to create bucket new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2 [2018-01-25 21:17:32,577] - [remote_util:232] INFO - connecting to 172.17.0.7 with username:root password:couchbase ssh_key: [2018-01-25 21:17:33,092] - [remote_util:266] INFO - Connected to 172.17.0.7 [2018-01-25 21:17:33,550] - [bucket_helper:251] INFO - waiting for bucket creation to complete.... [2018-01-25 21:17:33,558] - [rest_client:133] INFO - node 172.17.0.7 existing buckets : [u'new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2'] [2018-01-25 21:17:33,558] - [rest_client:137] INFO - node 172.17.0.7 found bucket new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2 [2018-01-25 21:17:34,124] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:17:34,126] - [rest_client:133] INFO - node 172.17.0.7 existing buckets : [] [2018-01-25 21:17:34,127] - [deletebuckettests:99] INFO - waiting for bucket data files deletion from the disk .... [2018-01-25 21:17:34,139] - [bucket_helper:174] INFO - deleting existing bucket new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2 on ip:172.17.0.7 port:8091 ssh_username:root [2018-01-25 21:17:34,148] - [rest_client:133] INFO - node 172.17.0.7 existing buckets : [] [2018-01-25 21:17:34,149] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:17:34,150] - [rest_client:133] INFO - node 172.17.0.7 existing buckets : [] [2018-01-25 21:17:34,161] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:17:34,161] - [rest_client:2237] INFO - http://172.17.0.5:8091/pools/default/buckets with param: bucketType=membase&threadsNumber=3&authType=none&replicaIndex=1&name=new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2&evictionPolicy=valueOnly&flushEnabled=1&replicaNumber=0&ramQuotaMB=200 [2018-01-25 21:17:34,166] - [rest_client:2262] INFO - 0.00 seconds to create bucket new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2 [2018-01-25 21:17:34,167] - [remote_util:232] INFO - connecting to 172.17.0.5 with username:root password:couchbase ssh_key: [2018-01-25 21:17:34,452] - [remote_util:266] INFO - Connected to 172.17.0.5 [2018-01-25 21:17:34,867] - [bucket_helper:251] INFO - waiting for bucket creation to complete.... [2018-01-25 21:17:34,873] - [rest_client:133] INFO - node 172.17.0.5 existing buckets : [u'new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2'] [2018-01-25 21:17:34,873] - [rest_client:137] INFO - node 172.17.0.5 found bucket new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2 [2018-01-25 21:17:35,441] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:17:35,450] - [rest_client:133] INFO - node 172.17.0.5 existing buckets : [] [2018-01-25 21:17:35,450] - [deletebuckettests:99] INFO - waiting for bucket data files deletion from the disk .... [2018-01-25 21:17:35,477] - [bucket_helper:174] INFO - deleting existing bucket new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2 on ip:172.17.0.5 port:8091 ssh_username:root [2018-01-25 21:17:35,492] - [rest_client:133] INFO - node 172.17.0.5 existing buckets : [] [2018-01-25 21:17:35,493] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:17:35,495] - [rest_client:133] INFO - node 172.17.0.5 existing buckets : [] [2018-01-25 21:17:35,519] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:17:35,520] - [rest_client:2237] INFO - http://172.17.0.5:8091/pools/default/buckets with param: bucketType=membase&threadsNumber=3&authType=none&replicaIndex=1&name=new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2&evictionPolicy=valueOnly&flushEnabled=1&replicaNumber=1&ramQuotaMB=200 [2018-01-25 21:17:35,535] - [rest_client:2262] INFO - 0.02 seconds to create bucket new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2 [2018-01-25 21:17:35,538] - [remote_util:232] INFO - connecting to 172.17.0.5 with username:root password:couchbase ssh_key: [2018-01-25 21:17:36,058] - [remote_util:266] INFO - Connected to 172.17.0.5 [2018-01-25 21:17:36,492] - [bucket_helper:251] INFO - waiting for bucket creation to complete.... [2018-01-25 21:17:36,502] - [rest_client:133] INFO - node 172.17.0.5 existing buckets : [u'new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2'] [2018-01-25 21:17:36,502] - [rest_client:137] INFO - node 172.17.0.5 found bucket new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2 [2018-01-25 21:17:37,542] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:17:37,544] - [rest_client:133] INFO - node 172.17.0.5 existing buckets : [] [2018-01-25 21:17:37,544] - [deletebuckettests:99] INFO - waiting for bucket data files deletion from the disk .... [2018-01-25 21:17:37,559] - [bucket_helper:174] INFO - deleting existing bucket new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2 on ip:172.17.0.5 port:8091 ssh_username:root [2018-01-25 21:17:37,565] - [rest_client:133] INFO - node 172.17.0.5 existing buckets : [] [2018-01-25 21:17:37,565] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:17:37,567] - [rest_client:133] INFO - node 172.17.0.5 existing buckets : [] [2018-01-25 21:17:37,577] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:17:37,578] - [rest_client:2237] INFO - http://172.17.0.5:8091/pools/default/buckets with param: bucketType=membase&threadsNumber=3&authType=none&replicaIndex=1&name=new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2&evictionPolicy=valueOnly&flushEnabled=1&replicaNumber=2&ramQuotaMB=200 [2018-01-25 21:17:37,586] - [rest_client:2262] INFO - 0.01 seconds to create bucket new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2 [2018-01-25 21:17:37,593] - [remote_util:232] INFO - connecting to 172.17.0.5 with username:root password:couchbase ssh_key: [2018-01-25 21:17:37,802] - [remote_util:266] INFO - Connected to 172.17.0.5 [2018-01-25 21:17:38,205] - [bucket_helper:251] INFO - waiting for bucket creation to complete.... [2018-01-25 21:17:38,211] - [rest_client:133] INFO - node 172.17.0.5 existing buckets : [u'new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2'] [2018-01-25 21:17:38,211] - [rest_client:137] INFO - node 172.17.0.5 found bucket new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2 [2018-01-25 21:17:38,864] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:17:38,868] - [rest_client:133] INFO - node 172.17.0.5 existing buckets : [] [2018-01-25 21:17:38,868] - [deletebuckettests:99] INFO - waiting for bucket data files deletion from the disk .... [2018-01-25 21:17:38,879] - [bucket_helper:174] INFO - deleting existing bucket new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2 on ip:172.17.0.5 port:8091 ssh_username:root [2018-01-25 21:17:38,885] - [rest_client:133] INFO - node 172.17.0.5 existing buckets : [] [2018-01-25 21:17:38,885] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:17:38,887] - [rest_client:133] INFO - node 172.17.0.5 existing buckets : [] [2018-01-25 21:17:38,898] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:17:38,898] - [rest_client:2237] INFO - http://172.17.0.5:8091/pools/default/buckets with param: bucketType=membase&threadsNumber=3&authType=none&replicaIndex=1&name=new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2&evictionPolicy=valueOnly&flushEnabled=1&replicaNumber=3&ramQuotaMB=200 [2018-01-25 21:17:38,904] - [rest_client:2262] INFO - 0.01 seconds to create bucket new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2 [2018-01-25 21:17:38,905] - [remote_util:232] INFO - connecting to 172.17.0.5 with username:root password:couchbase ssh_key: [2018-01-25 21:17:39,050] - [remote_util:266] INFO - Connected to 172.17.0.5 [2018-01-25 21:17:39,528] - [bucket_helper:251] INFO - waiting for bucket creation to complete.... [2018-01-25 21:17:39,533] - [rest_client:133] INFO - node 172.17.0.5 existing buckets : [u'new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2'] [2018-01-25 21:17:39,533] - [rest_client:137] INFO - node 172.17.0.5 found bucket new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2 [2018-01-25 21:17:40,175] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:17:40,178] - [rest_client:133] INFO - node 172.17.0.5 existing buckets : [] [2018-01-25 21:17:40,178] - [deletebuckettests:99] INFO - waiting for bucket data files deletion from the disk .... [2018-01-25 21:17:40,194] - [bucket_helper:174] INFO - deleting existing bucket new-bucket-19ca6a94-8049-4d54-b649-e2bb35c6dcb2 on ip:172.17.0.5 port:8091 ssh_username:root [2018-01-25 21:17:40,206] - [rest_client:133] INFO - node 172.17.0.5 existing buckets : [] [2018-01-25 21:17:40,206] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:17:40,211] - [rest_client:133] INFO - node 172.17.0.5 existing buckets : [] [2018-01-25 21:17:40,222] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.8 [2018-01-25 21:17:40,223] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:17:42,229] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.6 [2018-01-25 21:17:42,230] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:17:44,240] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.7 [2018-01-25 21:17:44,240] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:17:46,250] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.5 [2018-01-25 21:17:46,250] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. test_non_default_moxi (deletebuckettests.DeleteMembaseBuckets) ... ok ---------------------------------------------------------------------- Ran 1 test in 38.231s OK summary so far suite recreatebuckettests.RecreateMembaseBuckets , pass 0 , fail 1 failures so far... recreatebuckettests.RecreateMembaseBuckets.test_default_moxi summary so far suite deletebuckettests.DeleteMembaseBuckets , pass 1 , fail 0 testrunner logs, diags and results are available under /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_2 Logs will be stored at /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_3 ./testrunner -i ./4node.ini -t createbuckettests.CreateMembaseBucketsTests.test_default_moxi Test Input params: {'conf_file': './4node.conf', 'num_nodes': 4, 'cluster_name': '4node', 'ini': './4node.ini', 'case_number': 3, 'logs_folder': '/testrunner/logs/testrunner-18-Jan-25_21-16-28/test_3', 'spec': '4node'} [2018-01-25 21:17:48,272] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.8 [2018-01-25 21:17:48,273] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:17:50,281] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.6 [2018-01-25 21:17:50,282] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:17:52,290] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.7 [2018-01-25 21:17:52,290] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:17:54,298] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.5 [2018-01-25 21:17:54,298] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:17:56,322] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:17:56,322] - [rest_client:2237] INFO - http://172.17.0.8:8091/pools/default/buckets with param: bucketType=membase&threadsNumber=3&authType=sasl&replicaIndex=1&name=default&saslPassword=&evictionPolicy=valueOnly&flushEnabled=1&replicaNumber=1&ramQuotaMB=200 [2018-01-25 21:17:56,348] - [rest_client:2262] INFO - 0.03 seconds to create bucket default [2018-01-25 21:17:56,348] - [bucket_helper:251] INFO - waiting for bucket creation to complete.... [2018-01-25 21:17:56,375] - [rest_client:133] INFO - node 172.17.0.8 existing buckets : [u'default'] [2018-01-25 21:17:56,376] - [rest_client:137] INFO - node 172.17.0.8 found bucket default [2018-01-25 21:17:56,490] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:17:56,494] - [rest_client:2237] INFO - http://172.17.0.6:8091/pools/default/buckets with param: bucketType=membase&threadsNumber=3&authType=sasl&replicaIndex=1&name=default&saslPassword=&evictionPolicy=valueOnly&flushEnabled=1&replicaNumber=1&ramQuotaMB=200 [2018-01-25 21:17:56,503] - [rest_client:807] ERROR - POST http://172.17.0.6:8091/pools/default/buckets body: bucketType=membase&threadsNumber=3&authType=sasl&replicaIndex=1&name=default&saslPassword=&evictionPolicy=valueOnly&flushEnabled=1&replicaNumber=1&ramQuotaMB=200 headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 400 reason: unknown {"errors":{"ramQuotaMB":"RAM quota specified is too large to be provisioned into this cluster.","name":"Bucket with given name already exists"},"summaries":{"ramSummary":{"total":1073741824,"otherBuckets":838860800,"nodesCount":4,"perNodeMegs":200,"thisAlloc":838860800,"thisUsed":0,"free":-603979776},"hddSummary":{"total":69174132736,"otherData":11759602564,"otherBuckets":0,"thisUsed":0,"free":57414530172}}} auth: Administrator:password test_default_moxi (createbuckettests.CreateMembaseBucketsTests) ... ERROR [2018-01-25 21:17:56,534] - [bucket_helper:142] INFO - deleting existing buckets [u'default'] on 172.17.0.8 [2018-01-25 21:17:56,534] - [bucket_helper:144] INFO - remove bucket default ... [2018-01-25 21:17:57,037] - [bucket_helper:158] INFO - deleted bucket : default from 172.17.0.8 [2018-01-25 21:17:57,037] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:17:57,040] - [rest_client:133] INFO - node 172.17.0.8 existing buckets : [] [2018-01-25 21:17:57,040] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:17:59,049] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.6 [2018-01-25 21:17:59,049] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:18:01,060] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.7 [2018-01-25 21:18:01,060] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:18:03,069] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.5 [2018-01-25 21:18:03,069] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. ====================================================================== ERROR: test_default_moxi (createbuckettests.CreateMembaseBucketsTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "pytests/createbuckettests.py", line 55, in test_default_moxi rest.create_bucket(bucket=name, ramQuotaMB=200, proxyPort=proxyPort) File "lib/membase/api/rest_client.py", line 2250, in create_bucket raise BucketCreationException(ip=self.ip, bucket_name=bucket) BucketCreationException: unable to create bucket default on the host @ 172.17.0.6 ---------------------------------------------------------------------- Ran 1 test in 16.812s FAILED (errors=1) summary so far suite recreatebuckettests.RecreateMembaseBuckets , pass 0 , fail 1 failures so far... recreatebuckettests.RecreateMembaseBuckets.test_default_moxi summary so far suite deletebuckettests.DeleteMembaseBuckets , pass 1 , fail 0 summary so far suite createbuckettests.CreateMembaseBucketsTests , pass 0 , fail 1 failures so far... createbuckettests.CreateMembaseBucketsTests.test_default_moxi testrunner logs, diags and results are available under /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_3 Logs will be stored at /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_4 ./testrunner -i ./4node.ini -t createbuckettests.CreateMembaseBucketsTests.test_default_on_non_default_port Test Input params: {'conf_file': './4node.conf', 'num_nodes': 4, 'cluster_name': '4node', 'ini': './4node.ini', 'case_number': 4, 'logs_folder': '/testrunner/logs/testrunner-18-Jan-25_21-16-28/test_4', 'spec': '4node'} [2018-01-25 21:18:05,084] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.8 [2018-01-25 21:18:05,084] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:18:07,093] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.6 [2018-01-25 21:18:07,093] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:18:09,101] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.7 [2018-01-25 21:18:09,101] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:18:11,110] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.5 [2018-01-25 21:18:11,110] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:18:13,133] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:18:13,133] - [rest_client:2237] INFO - http://172.17.0.8:8091/pools/default/buckets with param: bucketType=membase&threadsNumber=3&authType=sasl&replicaIndex=1&name=default&saslPassword=test_non_default&evictionPolicy=valueOnly&flushEnabled=1&replicaNumber=1&ramQuotaMB=200 [2018-01-25 21:18:13,139] - [rest_client:2262] INFO - 0.01 seconds to create bucket default [2018-01-25 21:18:13,140] - [bucket_helper:251] INFO - waiting for bucket creation to complete.... [2018-01-25 21:18:13,343] - [rest_client:133] INFO - node 172.17.0.8 existing buckets : [u'default'] [2018-01-25 21:18:13,344] - [rest_client:137] INFO - node 172.17.0.8 found bucket default [2018-01-25 21:18:13,496] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:18:13,496] - [rest_client:2237] INFO - http://172.17.0.6:8091/pools/default/buckets with param: bucketType=membase&threadsNumber=3&authType=sasl&replicaIndex=1&name=default&saslPassword=test_non_default&evictionPolicy=valueOnly&flushEnabled=1&replicaNumber=1&ramQuotaMB=200 [2018-01-25 21:18:13,508] - [rest_client:807] ERROR - POST http://172.17.0.6:8091/pools/default/buckets body: bucketType=membase&threadsNumber=3&authType=sasl&replicaIndex=1&name=default&saslPassword=test_non_default&evictionPolicy=valueOnly&flushEnabled=1&replicaNumber=1&ramQuotaMB=200 headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 400 reason: unknown {"errors":{"ramQuotaMB":"RAM quota specified is too large to be provisioned into this cluster.","name":"Bucket with given name already exists"},"summaries":{"ramSummary":{"total":1073741824,"otherBuckets":838860800,"nodesCount":4,"perNodeMegs":200,"thisAlloc":838860800,"thisUsed":0,"free":-603979776},"hddSummary":{"total":69174132736,"otherData":11759602496,"otherBuckets":68,"thisUsed":0,"free":57414530172}}} auth: Administrator:password test_default_on_non_default_port (createbuckettests.CreateMembaseBucketsTests) ... ERROR [2018-01-25 21:18:13,535] - [bucket_helper:142] INFO - deleting existing buckets [u'default'] on 172.17.0.8 [2018-01-25 21:18:13,536] - [bucket_helper:144] INFO - remove bucket default ... [2018-01-25 21:18:15,297] - [bucket_helper:158] INFO - deleted bucket : default from 172.17.0.8 [2018-01-25 21:18:15,297] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:18:15,299] - [rest_client:133] INFO - node 172.17.0.8 existing buckets : [] [2018-01-25 21:18:15,299] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:18:17,309] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.6 [2018-01-25 21:18:17,309] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:18:19,318] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.7 [2018-01-25 21:18:19,318] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:18:21,326] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.5 [2018-01-25 21:18:21,326] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. ====================================================================== ERROR: test_default_on_non_default_port (createbuckettests.CreateMembaseBucketsTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "pytests/createbuckettests.py", line 96, in test_default_on_non_default_port saslPassword='test_non_default') File "lib/membase/api/rest_client.py", line 2250, in create_bucket raise BucketCreationException(ip=self.ip, bucket_name=bucket) BucketCreationException: unable to create bucket default on the host @ 172.17.0.6 ---------------------------------------------------------------------- Ran 1 test in 18.254s FAILED (errors=1) summary so far suite recreatebuckettests.RecreateMembaseBuckets , pass 0 , fail 1 failures so far... recreatebuckettests.RecreateMembaseBuckets.test_default_moxi summary so far suite deletebuckettests.DeleteMembaseBuckets , pass 1 , fail 0 summary so far suite createbuckettests.CreateMembaseBucketsTests , pass 0 , fail 2 failures so far... createbuckettests.CreateMembaseBucketsTests.test_default_moxi createbuckettests.CreateMembaseBucketsTests.test_default_on_non_default_port testrunner logs, diags and results are available under /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_4 Logs will be stored at /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_5 ./testrunner -i ./4node.ini -t createbuckettests.CreateMembaseBucketsTests.test_non_default_case_sensitive_different_port Test Input params: {'conf_file': './4node.conf', 'num_nodes': 4, 'cluster_name': '4node', 'ini': './4node.ini', 'case_number': 5, 'logs_folder': '/testrunner/logs/testrunner-18-Jan-25_21-16-28/test_5', 'spec': '4node'} [2018-01-25 21:18:23,346] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.8 [2018-01-25 21:18:23,347] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:18:25,352] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.6 [2018-01-25 21:18:25,352] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:18:27,360] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.7 [2018-01-25 21:18:27,360] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:18:29,366] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.5 [2018-01-25 21:18:29,366] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:18:31,386] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:18:31,386] - [rest_client:2237] INFO - http://172.17.0.8:8091/pools/default/buckets with param: bucketType=membase&threadsNumber=3&authType=none&replicaIndex=1&name=uppercase_991e304c-f2f0-4c5d-aa77-1d0a7f10d217&evictionPolicy=valueOnly&flushEnabled=1&replicaNumber=1&ramQuotaMB=200 [2018-01-25 21:18:31,393] - [rest_client:2262] INFO - 0.01 seconds to create bucket uppercase_991e304c-f2f0-4c5d-aa77-1d0a7f10d217 [2018-01-25 21:18:31,393] - [bucket_helper:251] INFO - waiting for bucket creation to complete.... [2018-01-25 21:18:31,397] - [rest_client:133] INFO - node 172.17.0.8 existing buckets : [] [2018-01-25 21:18:33,425] - [rest_client:133] INFO - node 172.17.0.8 existing buckets : [u'uppercase_991e304c-f2f0-4c5d-aa77-1d0a7f10d217'] [2018-01-25 21:18:33,425] - [rest_client:137] INFO - node 172.17.0.8 found bucket uppercase_991e304c-f2f0-4c5d-aa77-1d0a7f10d217 [2018-01-25 21:18:33,492] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:18:33,498] - [rest_client:2237] INFO - http://172.17.0.8:8091/pools/default/buckets with param: bucketType=membase&threadsNumber=3&authType=none&replicaIndex=1&name=UPPERCASE_991e304c-f2f0-4c5d-aa77-1d0a7f10d217&evictionPolicy=valueOnly&flushEnabled=1&replicaNumber=1&ramQuotaMB=200 [2018-01-25 21:18:33,589] - [rest_client:807] ERROR - POST http://172.17.0.8:8091/pools/default/buckets body: bucketType=membase&threadsNumber=3&authType=none&replicaIndex=1&name=UPPERCASE_991e304c-f2f0-4c5d-aa77-1d0a7f10d217&evictionPolicy=valueOnly&flushEnabled=1&replicaNumber=1&ramQuotaMB=200 headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 400 reason: unknown {"errors":{"ramQuotaMB":"RAM quota specified is too large to be provisioned into this cluster.","name":"Bucket with given name already exists"},"summaries":{"ramSummary":{"total":1073741824,"otherBuckets":838860800,"nodesCount":4,"perNodeMegs":200,"thisAlloc":838860800,"thisUsed":0,"free":-603979776},"hddSummary":{"total":69174132736,"otherData":11759602564,"otherBuckets":0,"thisUsed":0,"free":57414530172}}} auth: Administrator:password [2018-01-25 21:18:33,590] - [createbuckettests:156] INFO - BucketCreationException was thrown as expected [2018-01-25 21:18:33,590] - [createbuckettests:157] INFO - [2018-01-25 21:18:33,769] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:18:33,770] - [rest_client:2237] INFO - http://172.17.0.6:8091/pools/default/buckets with param: bucketType=membase&threadsNumber=3&authType=none&replicaIndex=1&name=uppercase_991e304c-f2f0-4c5d-aa77-1d0a7f10d217&evictionPolicy=valueOnly&flushEnabled=1&replicaNumber=1&ramQuotaMB=200 [2018-01-25 21:18:33,817] - [rest_client:807] ERROR - POST http://172.17.0.6:8091/pools/default/buckets body: bucketType=membase&threadsNumber=3&authType=none&replicaIndex=1&name=uppercase_991e304c-f2f0-4c5d-aa77-1d0a7f10d217&evictionPolicy=valueOnly&flushEnabled=1&replicaNumber=1&ramQuotaMB=200 headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 400 reason: unknown {"errors":{"ramQuotaMB":"RAM quota specified is too large to be provisioned into this cluster.","name":"Bucket with given name already exists"},"summaries":{"ramSummary":{"total":1073741824,"otherBuckets":838860800,"nodesCount":4,"perNodeMegs":200,"thisAlloc":838860800,"thisUsed":0,"free":-603979776},"hddSummary":{"total":69174132736,"otherData":11759602564,"otherBuckets":0,"thisUsed":0,"free":57414530172}}} auth: Administrator:password test_non_default_case_sensitive_different_port (createbuckettests.CreateMembaseBucketsTests) ... ERROR [2018-01-25 21:18:33,879] - [bucket_helper:142] INFO - deleting existing buckets [u'uppercase_991e304c-f2f0-4c5d-aa77-1d0a7f10d217'] on 172.17.0.8 [2018-01-25 21:18:33,879] - [bucket_helper:144] INFO - remove bucket uppercase_991e304c-f2f0-4c5d-aa77-1d0a7f10d217 ... [2018-01-25 21:18:34,859] - [bucket_helper:158] INFO - deleted bucket : uppercase_991e304c-f2f0-4c5d-aa77-1d0a7f10d217 from 172.17.0.8 [2018-01-25 21:18:34,859] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:18:34,862] - [rest_client:133] INFO - node 172.17.0.8 existing buckets : [] [2018-01-25 21:18:34,862] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:18:36,875] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.6 [2018-01-25 21:18:36,875] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:18:38,881] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.7 [2018-01-25 21:18:38,882] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:18:40,891] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.5 [2018-01-25 21:18:40,891] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. ====================================================================== ERROR: test_non_default_case_sensitive_different_port (createbuckettests.CreateMembaseBucketsTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "pytests/createbuckettests.py", line 142, in test_non_default_case_sensitive_different_port proxyPort=proxyPort) File "lib/membase/api/rest_client.py", line 2250, in create_bucket raise BucketCreationException(ip=self.ip, bucket_name=bucket) BucketCreationException: unable to create bucket uppercase_991e304c-f2f0-4c5d-aa77-1d0a7f10d217 on the host @ 172.17.0.6 ---------------------------------------------------------------------- Ran 1 test in 19.563s FAILED (errors=1) summary so far suite recreatebuckettests.RecreateMembaseBuckets , pass 0 , fail 1 failures so far... recreatebuckettests.RecreateMembaseBuckets.test_default_moxi summary so far suite deletebuckettests.DeleteMembaseBuckets , pass 1 , fail 0 summary so far suite createbuckettests.CreateMembaseBucketsTests , pass 0 , fail 3 failures so far... createbuckettests.CreateMembaseBucketsTests.test_default_moxi createbuckettests.CreateMembaseBucketsTests.test_default_on_non_default_port createbuckettests.CreateMembaseBucketsTests.test_non_default_case_sensitive_different_port testrunner logs, diags and results are available under /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_5 Logs will be stored at /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_6 ./testrunner -i ./4node.ini -t createbuckettests.CreateMembaseBucketsTests.test_two_replica Test Input params: {'conf_file': './4node.conf', 'num_nodes': 4, 'cluster_name': '4node', 'ini': './4node.ini', 'case_number': 6, 'logs_folder': '/testrunner/logs/testrunner-18-Jan-25_21-16-28/test_6', 'spec': '4node'} [2018-01-25 21:18:42,912] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.8 [2018-01-25 21:18:42,912] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:18:44,922] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.6 [2018-01-25 21:18:44,922] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:18:46,930] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.7 [2018-01-25 21:18:46,930] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:18:48,939] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.5 [2018-01-25 21:18:48,940] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:18:50,959] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:18:50,959] - [rest_client:2237] INFO - http://172.17.0.8:8091/pools/default/buckets with param: bucketType=membase&threadsNumber=3&authType=sasl&replicaIndex=1&name=2replica_425233c7-fde8-4f2f-94b4-6a9e15568677&saslPassword=&evictionPolicy=valueOnly&flushEnabled=1&replicaNumber=2&ramQuotaMB=200 [2018-01-25 21:18:50,971] - [rest_client:2262] INFO - 0.01 seconds to create bucket 2replica_425233c7-fde8-4f2f-94b4-6a9e15568677 [2018-01-25 21:18:50,972] - [bucket_helper:251] INFO - waiting for bucket creation to complete.... [2018-01-25 21:18:51,122] - [rest_client:133] INFO - node 172.17.0.8 existing buckets : [u'2replica_425233c7-fde8-4f2f-94b4-6a9e15568677'] [2018-01-25 21:18:51,166] - [rest_client:137] INFO - node 172.17.0.8 found bucket 2replica_425233c7-fde8-4f2f-94b4-6a9e15568677 [2018-01-25 21:18:51,367] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:18:51,368] - [rest_client:2237] INFO - http://172.17.0.6:8091/pools/default/buckets with param: bucketType=membase&threadsNumber=3&authType=sasl&replicaIndex=1&name=2replica_425233c7-fde8-4f2f-94b4-6a9e15568677&saslPassword=&evictionPolicy=valueOnly&flushEnabled=1&replicaNumber=2&ramQuotaMB=200 [2018-01-25 21:18:51,486] - [rest_client:807] ERROR - POST http://172.17.0.6:8091/pools/default/buckets body: bucketType=membase&threadsNumber=3&authType=sasl&replicaIndex=1&name=2replica_425233c7-fde8-4f2f-94b4-6a9e15568677&saslPassword=&evictionPolicy=valueOnly&flushEnabled=1&replicaNumber=2&ramQuotaMB=200 headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 400 reason: unknown {"errors":{"ramQuotaMB":"RAM quota specified is too large to be provisioned into this cluster.","name":"Bucket with given name already exists"},"summaries":{"ramSummary":{"total":1073741824,"otherBuckets":838860800,"nodesCount":4,"perNodeMegs":200,"thisAlloc":838860800,"thisUsed":0,"free":-603979776},"hddSummary":{"total":69174132736,"otherData":11759602564,"otherBuckets":0,"thisUsed":0,"free":57414530172}}} auth: Administrator:password [2018-01-25 21:18:51,497] - [createbuckettests:287] ERROR - unable to create bucket 2replica_425233c7-fde8-4f2f-94b4-6a9e15568677 on the host @ 172.17.0.6 test_two_replica (createbuckettests.CreateMembaseBucketsTests) ... FAIL [2018-01-25 21:18:51,563] - [bucket_helper:142] INFO - deleting existing buckets [u'2replica_425233c7-fde8-4f2f-94b4-6a9e15568677'] on 172.17.0.8 [2018-01-25 21:18:51,563] - [bucket_helper:144] INFO - remove bucket 2replica_425233c7-fde8-4f2f-94b4-6a9e15568677 ... [2018-01-25 21:18:53,227] - [bucket_helper:158] INFO - deleted bucket : 2replica_425233c7-fde8-4f2f-94b4-6a9e15568677 from 172.17.0.8 [2018-01-25 21:18:53,227] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:18:53,231] - [rest_client:133] INFO - node 172.17.0.8 existing buckets : [] [2018-01-25 21:18:53,231] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:18:55,240] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.6 [2018-01-25 21:18:55,240] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:18:57,252] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.7 [2018-01-25 21:18:57,252] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:18:59,269] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.5 [2018-01-25 21:18:59,269] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. ====================================================================== FAIL: test_two_replica (createbuckettests.CreateMembaseBucketsTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "pytests/createbuckettests.py", line 288, in test_two_replica self.fail('failed to create bucket with 2 replicas') AssertionError: failed to create bucket with 2 replicas ---------------------------------------------------------------------- Ran 1 test in 18.375s FAILED (failures=1) summary so far suite recreatebuckettests.RecreateMembaseBuckets , pass 0 , fail 1 failures so far... recreatebuckettests.RecreateMembaseBuckets.test_default_moxi summary so far suite deletebuckettests.DeleteMembaseBuckets , pass 1 , fail 0 summary so far suite createbuckettests.CreateMembaseBucketsTests , pass 0 , fail 4 failures so far... createbuckettests.CreateMembaseBucketsTests.test_default_moxi createbuckettests.CreateMembaseBucketsTests.test_default_on_non_default_port createbuckettests.CreateMembaseBucketsTests.test_non_default_case_sensitive_different_port createbuckettests.CreateMembaseBucketsTests.test_two_replica testrunner logs, diags and results are available under /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_6 Logs will be stored at /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_7 ./testrunner -i ./4node.ini -t createbuckettests.CreateMembaseBucketsTests.test_valid_length,name_length=100 Test Input params: {'cluster_name': '4node', 'name_length': '100', 'ini': './4node.ini', 'case_number': 7, 'logs_folder': '/testrunner/logs/testrunner-18-Jan-25_21-16-28/test_7', 'spec': '4node', 'conf_file': './4node.conf', 'num_nodes': 4} [2018-01-25 21:19:01,288] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.8 [2018-01-25 21:19:01,288] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:19:03,297] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.6 [2018-01-25 21:19:03,298] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:19:05,306] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.7 [2018-01-25 21:19:05,306] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:19:07,316] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.5 [2018-01-25 21:19:07,316] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:19:09,339] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:19:09,339] - [rest_client:2237] INFO - http://172.17.0.8:8091/pools/default/buckets with param: bucketType=membase&threadsNumber=3&authType=sasl&replicaIndex=1&name=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa&saslPassword=&evictionPolicy=valueOnly&flushEnabled=1&replicaNumber=1&ramQuotaMB=200 [2018-01-25 21:19:09,364] - [rest_client:2262] INFO - 0.02 seconds to create bucket aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa [2018-01-25 21:19:09,364] - [bucket_helper:251] INFO - waiting for bucket creation to complete.... [2018-01-25 21:19:09,412] - [rest_client:133] INFO - node 172.17.0.8 existing buckets : [u'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa'] [2018-01-25 21:19:09,413] - [rest_client:137] INFO - node 172.17.0.8 found bucket aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa [2018-01-25 21:19:09,687] - [bucket_helper:142] INFO - deleting existing buckets [u'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa'] on 172.17.0.8 [2018-01-25 21:19:09,687] - [bucket_helper:144] INFO - remove bucket aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa ... [2018-01-25 21:19:10,541] - [bucket_helper:158] INFO - deleted bucket : aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa from 172.17.0.8 [2018-01-25 21:19:10,541] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:19:10,544] - [rest_client:133] INFO - node 172.17.0.8 existing buckets : [] [2018-01-25 21:19:10,544] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:19:12,550] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.6 [2018-01-25 21:19:12,551] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:19:14,559] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.7 [2018-01-25 21:19:14,560] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:19:16,568] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.5 [2018-01-25 21:19:16,568] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. test_valid_length (createbuckettests.CreateMembaseBucketsTests) ... ok ---------------------------------------------------------------------- Ran 1 test in 17.293s OK summary so far suite recreatebuckettests.RecreateMembaseBuckets , pass 0 , fail 1 failures so far... recreatebuckettests.RecreateMembaseBuckets.test_default_moxi summary so far suite deletebuckettests.DeleteMembaseBuckets , pass 1 , fail 0 summary so far suite createbuckettests.CreateMembaseBucketsTests , pass 1 , fail 4 failures so far... createbuckettests.CreateMembaseBucketsTests.test_default_moxi createbuckettests.CreateMembaseBucketsTests.test_default_on_non_default_port createbuckettests.CreateMembaseBucketsTests.test_non_default_case_sensitive_different_port createbuckettests.CreateMembaseBucketsTests.test_two_replica testrunner logs, diags and results are available under /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_7 Logs will be stored at /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_8 ./testrunner -i ./4node.ini -t setgettests.MembaseBucket.test_value_100b Test Input params: {'conf_file': './4node.conf', 'num_nodes': 4, 'cluster_name': '4node', 'ini': './4node.ini', 'case_number': 8, 'logs_folder': '/testrunner/logs/testrunner-18-Jan-25_21-16-28/test_8', 'spec': '4node'} [2018-01-25 21:19:18,645] - [rest_client:825] INFO - settings/web params on 172.17.0.8:8091:username=Administrator&password=password&port=8091 [2018-01-25 21:19:18,739] - [rest_client:907] INFO - pools/default params : memoryQuota=6234 [2018-01-25 21:19:18,805] - [cluster_helper:254] INFO - rebalancing all nodes in order to remove nodes [2018-01-25 21:19:18,817] - [rest_client:1423] INFO - rebalance params : {'password': 'password', 'ejectedNodes': u'ns_1@cb-example-0001.cb-example.default.svc,ns_1@cb-example-0000.cb-example.default.svc,ns_1@cb-example-0002.cb-example.default.svc', 'user': 'Administrator', 'knownNodes': u'ns_1@cb-example-0001.cb-example.default.svc,ns_1@cb-example-0000.cb-example.default.svc,ns_1@cb-example-0003.cb-example.default.svc,ns_1@cb-example-0002.cb-example.default.svc'} [2018-01-25 21:19:18,829] - [rest_client:1428] INFO - rebalance operation started [2018-01-25 21:19:18,861] - [rest_client:1576] INFO - rebalance percentage : 0.00 % [2018-01-25 21:19:28,864] - [rest_client:1576] INFO - rebalance percentage : 33.33 % [2018-01-25 21:19:38,880] - [rest_client:1576] INFO - rebalance percentage : 33.33 % [2018-01-25 21:19:58,906] - [rest_client:1515] INFO - rebalance progress took 40.0766108036 seconds [2018-01-25 21:19:58,906] - [rest_client:1516] INFO - sleep for 10 seconds after rebalance... [2018-01-25 21:20:08,931] - [cluster_helper:294] INFO - removed all the nodes from cluster associated with ip:172.17.0.8 port:8091 ssh_username:root ? [(u'ns_1@cb-example-0001.cb-example.default.svc', 8091), (u'ns_1@cb-example-0000.cb-example.default.svc', 8091), (u'ns_1@cb-example-0002.cb-example.default.svc', 8091)] [2018-01-25 21:20:08,936] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.8 [2018-01-25 21:20:08,936] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:20:10,943] - [rest_client:825] INFO - settings/web params on 172.17.0.8:8091:username=Administrator&password=password&port=8091 [2018-01-25 21:20:10,975] - [rest_client:907] INFO - pools/default params : memoryQuota=6234 [2018-01-25 21:20:11,002] - [rest_client:807] ERROR - DELETE http://172.17.0.8:8091/settings/rbac/users/local/cbadminbucket body: headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 404 reason: unknown "User was not found." auth: Administrator:password [2018-01-25 21:20:11,003] - [internal_user:36] INFO - Exception while deleting user. Exception is -"User was not found." [2018-01-25 21:20:31,077] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise'] [2018-01-25 21:20:31,077] - [rest_client:2237] INFO - http://172.17.0.8:8091/pools/default/buckets with param: bucketType=membase&threadsNumber=3&authType=sasl&replicaIndex=1&name=default&saslPassword=password&evictionPolicy=valueOnly&flushEnabled=1&replicaNumber=0&ramQuotaMB=1558 [2018-01-25 21:20:31,081] - [rest_client:2262] INFO - 0.00 seconds to create bucket default [2018-01-25 21:20:31,119] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise'] [2018-01-25 21:20:31,120] - [rest_client:2237] INFO - http://172.17.0.8:8091/pools/default/buckets with param: bucketType=membase&threadsNumber=3&authType=sasl&replicaIndex=1&name=set-get-bucket-replica-1&saslPassword=password&evictionPolicy=valueOnly&flushEnabled=1&replicaNumber=1&ramQuotaMB=1558 [2018-01-25 21:20:31,152] - [rest_client:2262] INFO - 0.03 seconds to create bucket set-get-bucket-replica-1 [2018-01-25 21:20:31,170] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise'] [2018-01-25 21:20:31,170] - [rest_client:2237] INFO - http://172.17.0.8:8091/pools/default/buckets with param: bucketType=membase&threadsNumber=3&authType=sasl&replicaIndex=1&name=set-get-bucket-replica-2&saslPassword=password&evictionPolicy=valueOnly&flushEnabled=1&replicaNumber=2&ramQuotaMB=1558 [2018-01-25 21:20:31,182] - [rest_client:2262] INFO - 0.01 seconds to create bucket set-get-bucket-replica-2 [2018-01-25 21:20:31,195] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise'] [2018-01-25 21:20:31,196] - [rest_client:2237] INFO - http://172.17.0.8:8091/pools/default/buckets with param: bucketType=membase&threadsNumber=3&authType=sasl&replicaIndex=1&name=set-get-bucket-replica-3&saslPassword=password&evictionPolicy=valueOnly&flushEnabled=1&replicaNumber=3&ramQuotaMB=1558 [2018-01-25 21:20:31,207] - [rest_client:2262] INFO - 0.01 seconds to create bucket set-get-bucket-replica-3 [2018-01-25 21:20:31,296] - [bucket_helper:343] INFO - waiting for memcached bucket : default in 172.17.0.8 to accept set ops [2018-01-25 21:20:31,388] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:20:31,709] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:20:33,306] - [data_helper:295] INFO - creating direct client 172.17.0.8:11210 default [2018-01-25 21:20:36,935] - [bucket_helper:343] INFO - waiting for memcached bucket : set-get-bucket-replica-1 in 172.17.0.8 to accept set ops [2018-01-25 21:20:37,005] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 set-get-bucket-replica-1 [2018-01-25 21:20:37,152] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 set-get-bucket-replica-1 [2018-01-25 21:20:37,486] - [data_helper:295] INFO - creating direct client 172.17.0.8:11210 set-get-bucket-replica-1 [2018-01-25 21:20:41,894] - [bucket_helper:343] INFO - waiting for memcached bucket : set-get-bucket-replica-2 in 172.17.0.8 to accept set ops [2018-01-25 21:20:41,975] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 set-get-bucket-replica-2 [2018-01-25 21:20:42,048] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 set-get-bucket-replica-2 [2018-01-25 21:20:42,273] - [data_helper:295] INFO - creating direct client 172.17.0.8:11210 set-get-bucket-replica-2 [2018-01-25 21:20:45,737] - [bucket_helper:343] INFO - waiting for memcached bucket : set-get-bucket-replica-3 in 172.17.0.8 to accept set ops [2018-01-25 21:20:45,809] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 set-get-bucket-replica-3 [2018-01-25 21:20:45,887] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 set-get-bucket-replica-3 [2018-01-25 21:20:46,107] - [data_helper:295] INFO - creating direct client 172.17.0.8:11210 set-get-bucket-replica-3 [2018-01-25 21:20:49,625] - [bucket_helper:142] INFO - deleting existing buckets [u'default', u'set-get-bucket-replica-1', u'set-get-bucket-replica-2', u'set-get-bucket-replica-3'] on 172.17.0.8 [2018-01-25 21:20:49,626] - [bucket_helper:144] INFO - remove bucket default ... [2018-01-25 21:20:50,176] - [bucket_helper:158] INFO - deleted bucket : default from 172.17.0.8 [2018-01-25 21:20:50,176] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:20:50,198] - [rest_client:133] INFO - node 172.17.0.8 existing buckets : [u'set-get-bucket-replica-1', u'set-get-bucket-replica-2', u'set-get-bucket-replica-3'] [2018-01-25 21:20:50,199] - [bucket_helper:144] INFO - remove bucket set-get-bucket-replica-1 ... [2018-01-25 21:20:50,727] - [bucket_helper:158] INFO - deleted bucket : set-get-bucket-replica-1 from 172.17.0.8 [2018-01-25 21:20:50,727] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:20:50,751] - [rest_client:133] INFO - node 172.17.0.8 existing buckets : [u'set-get-bucket-replica-2', u'set-get-bucket-replica-3'] [2018-01-25 21:20:50,752] - [bucket_helper:144] INFO - remove bucket set-get-bucket-replica-2 ... [2018-01-25 21:20:51,405] - [bucket_helper:158] INFO - deleted bucket : set-get-bucket-replica-2 from 172.17.0.8 [2018-01-25 21:20:51,406] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:20:51,415] - [rest_client:133] INFO - node 172.17.0.8 existing buckets : [u'set-get-bucket-replica-3'] [2018-01-25 21:20:51,416] - [bucket_helper:144] INFO - remove bucket set-get-bucket-replica-3 ... [2018-01-25 21:20:51,849] - [bucket_helper:158] INFO - deleted bucket : set-get-bucket-replica-3 from 172.17.0.8 [2018-01-25 21:20:51,849] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:20:51,852] - [rest_client:133] INFO - node 172.17.0.8 existing buckets : [] [2018-01-25 21:20:51,852] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. test_value_100b (setgettests.MembaseBucket) ... ok ---------------------------------------------------------------------- Ran 1 test in 95.227s OK summary so far suite recreatebuckettests.RecreateMembaseBuckets , pass 0 , fail 1 failures so far... recreatebuckettests.RecreateMembaseBuckets.test_default_moxi summary so far suite deletebuckettests.DeleteMembaseBuckets , pass 1 , fail 0 summary so far suite createbuckettests.CreateMembaseBucketsTests , pass 1 , fail 4 failures so far... createbuckettests.CreateMembaseBucketsTests.test_default_moxi createbuckettests.CreateMembaseBucketsTests.test_default_on_non_default_port createbuckettests.CreateMembaseBucketsTests.test_non_default_case_sensitive_different_port createbuckettests.CreateMembaseBucketsTests.test_two_replica summary so far suite setgettests.MembaseBucket , pass 1 , fail 0 testrunner logs, diags and results are available under /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_8 Logs will be stored at /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_9 ./testrunner -i ./4node.ini -t expirytests.ExpiryTests.test_expired_keys Test Input params: {'conf_file': './4node.conf', 'num_nodes': 4, 'cluster_name': '4node', 'ini': './4node.ini', 'case_number': 9, 'logs_folder': '/testrunner/logs/testrunner-18-Jan-25_21-16-28/test_9', 'spec': '4node'} [2018-01-25 21:20:53,913] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.8 [2018-01-25 21:20:53,913] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:20:55,925] - [rest_client:825] INFO - settings/web params on 172.17.0.8:8091:username=Administrator&password=password&port=8091 [2018-01-25 21:20:55,960] - [rest_client:907] INFO - pools/default params : memoryQuota=6234 [2018-01-25 21:20:55,991] - [rest_client:807] ERROR - DELETE http://172.17.0.8:8091/settings/rbac/users/local/cbadminbucket body: headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 404 reason: unknown "User was not found." auth: Administrator:password [2018-01-25 21:20:55,991] - [internal_user:36] INFO - Exception while deleting user. Exception is -"User was not found." [2018-01-25 21:21:16,071] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise'] [2018-01-25 21:21:16,072] - [rest_client:2237] INFO - http://172.17.0.8:8091/pools/default/buckets with param: bucketType=membase&threadsNumber=3&authType=sasl&replicaIndex=1&name=default&saslPassword=&evictionPolicy=valueOnly&flushEnabled=1&replicaNumber=1&ramQuotaMB=4156 [2018-01-25 21:21:16,076] - [rest_client:2262] INFO - 0.00 seconds to create bucket default [2018-01-25 21:21:16,076] - [bucket_helper:251] INFO - waiting for bucket creation to complete.... [2018-01-25 21:21:16,085] - [rest_client:133] INFO - node 172.17.0.8 existing buckets : [u'default'] [2018-01-25 21:21:16,086] - [rest_client:137] INFO - node 172.17.0.8 found bucket default [2018-01-25 21:21:16,086] - [bucket_helper:343] INFO - waiting for memcached bucket : default in 172.17.0.8 to accept set ops [2018-01-25 21:21:17,257] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:21:17,484] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:21:19,182] - [data_helper:295] INFO - creating direct client 172.17.0.8:11210 default [2018-01-25 21:21:19,214] - [expirytests:85] INFO - pushing keys with expiry set to 2 [2018-01-25 21:21:19,447] - [expirytests:93] INFO - inserted 500 keys with expiry set to 2 [2018-01-25 21:21:19,448] - [expirytests:96] INFO - sleeping for 7 seconds to wait for those items with expiry set to 2 to expire [2018-01-25 21:21:26,452] - [expirytests:98] INFO - verifying that all those keys have expired... [2018-01-25 21:21:26,505] - [expirytests:107] INFO - verified that those keys inserted with expiry set to 2 have expired [2018-01-25 21:21:26,507] - [expirytests:85] INFO - pushing keys with expiry set to 5 [2018-01-25 21:21:26,557] - [expirytests:93] INFO - inserted 500 keys with expiry set to 5 [2018-01-25 21:21:26,558] - [expirytests:96] INFO - sleeping for 10 seconds to wait for those items with expiry set to 5 to expire [2018-01-25 21:21:36,561] - [expirytests:98] INFO - verifying that all those keys have expired... [2018-01-25 21:21:36,843] - [expirytests:107] INFO - verified that those keys inserted with expiry set to 5 have expired [2018-01-25 21:21:36,844] - [expirytests:85] INFO - pushing keys with expiry set to 10 [2018-01-25 21:21:37,145] - [expirytests:93] INFO - inserted 500 keys with expiry set to 10 [2018-01-25 21:21:37,146] - [expirytests:96] INFO - sleeping for 15 seconds to wait for those items with expiry set to 10 to expire [2018-01-25 21:21:52,147] - [expirytests:98] INFO - verifying that all those keys have expired... [2018-01-25 21:21:52,214] - [expirytests:107] INFO - verified that those keys inserted with expiry set to 10 have expired [2018-01-25 21:21:52,232] - [bucket_helper:142] INFO - deleting existing buckets [u'default'] on 172.17.0.8 [2018-01-25 21:21:52,232] - [bucket_helper:144] INFO - remove bucket default ... [2018-01-25 21:21:52,864] - [bucket_helper:158] INFO - deleted bucket : default from 172.17.0.8 [2018-01-25 21:21:52,865] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:21:52,867] - [rest_client:133] INFO - node 172.17.0.8 existing buckets : [] [2018-01-25 21:21:52,867] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. test_expired_keys (expirytests.ExpiryTests) ... ok ---------------------------------------------------------------------- Ran 1 test in 61.003s OK summary so far suite recreatebuckettests.RecreateMembaseBuckets , pass 0 , fail 1 failures so far... recreatebuckettests.RecreateMembaseBuckets.test_default_moxi summary so far suite deletebuckettests.DeleteMembaseBuckets , pass 1 , fail 0 summary so far suite createbuckettests.CreateMembaseBucketsTests , pass 1 , fail 4 failures so far... createbuckettests.CreateMembaseBucketsTests.test_default_moxi createbuckettests.CreateMembaseBucketsTests.test_default_on_non_default_port createbuckettests.CreateMembaseBucketsTests.test_non_default_case_sensitive_different_port createbuckettests.CreateMembaseBucketsTests.test_two_replica summary so far suite setgettests.MembaseBucket , pass 1 , fail 0 summary so far suite expirytests.ExpiryTests , pass 1 , fail 0 testrunner logs, diags and results are available under /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_9 Logs will be stored at /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_10 ./testrunner -i ./4node.ini -t memcapable.GetlTests.test_getl_expired_item Test Input params: {'conf_file': './4node.conf', 'num_nodes': 4, 'cluster_name': '4node', 'ini': './4node.ini', 'case_number': 10, 'logs_folder': '/testrunner/logs/testrunner-18-Jan-25_21-16-28/test_10', 'spec': '4node'} [2018-01-25 21:21:54,975] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.8 [2018-01-25 21:21:54,975] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:21:56,992] - [rest_client:807] ERROR - DELETE http://172.17.0.8:8091/settings/rbac/users/local/cbadminbucket body: headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 404 reason: unknown "User was not found." auth: Administrator:password [2018-01-25 21:21:56,992] - [internal_user:36] INFO - Exception while deleting user. Exception is -"User was not found." [2018-01-25 21:22:17,073] - [rest_client:133] INFO - node 172.17.0.8 existing buckets : [] [2018-01-25 21:22:17,084] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise'] [2018-01-25 21:22:17,084] - [rest_client:2237] INFO - http://172.17.0.8:8091/pools/default/buckets with param: bucketType=membase&threadsNumber=3&authType=sasl&replicaIndex=1&name=default&saslPassword=&evictionPolicy=valueOnly&flushEnabled=1&replicaNumber=1&ramQuotaMB=4156 [2018-01-25 21:22:17,091] - [rest_client:2262] INFO - 0.01 seconds to create bucket default [2018-01-25 21:22:17,091] - [bucket_helper:343] INFO - waiting for memcached bucket : default in 172.17.0.8 to accept set ops [2018-01-25 21:22:18,388] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:22:18,733] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:22:20,011] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:22:20,070] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:22:20,261] - [rest_client:133] INFO - node 172.17.0.8 existing buckets : [u'default'] [2018-01-25 21:22:20,261] - [rest_client:137] INFO - node 172.17.0.8 found bucket default [2018-01-25 21:22:40,350] - [data_helper:295] INFO - creating direct client 172.17.0.8:11210 default [2018-01-25 21:22:40,379] - [memcapable:353] INFO - setting key getl_expired_item_a3632572-48b4-475b-885e-70a981de113b with expiration 5 [2018-01-25 21:22:40,380] - [memcapable:355] INFO - getl key getl_expired_item_a3632572-48b4-475b-885e-70a981de113b timeout 15 [2018-01-25 21:22:40,381] - [memcapable:357] INFO - get key getl_expired_item_a3632572-48b4-475b-885e-70a981de113b which is locked now [2018-01-25 21:22:40,381] - [memcapable:364] INFO - sleep for 15 seconds [2018-01-25 21:22:55,397] - [memcapable:369] INFO - get key getl_expired_item_a3632572-48b4-475b-885e-70a981de113b which was locked when it expired. should fail [2018-01-25 21:22:55,398] - [memcapable:374] INFO - raised exception as expected : Memcached error #1 'Not found': for vbucket :852 to mc 172.17.0.8:11210 [2018-01-25 21:22:55,398] - [memcapable:375] INFO - item expired and lock should have timed out by now . try to set the item again [2018-01-25 21:22:55,399] - [memcapable:377] INFO - setting key getl_expired_item_a3632572-48b4-475b-885e-70a981de113b with new value * [2018-01-25 21:22:55,399] - [memcapable:379] INFO - get key getl_expired_item_a3632572-48b4-475b-885e-70a981de113b [2018-01-25 21:22:55,428] - [bucket_helper:142] INFO - deleting existing buckets [u'default'] on 172.17.0.8 [2018-01-25 21:22:55,428] - [bucket_helper:144] INFO - remove bucket default ... [2018-01-25 21:22:55,864] - [bucket_helper:158] INFO - deleted bucket : default from 172.17.0.8 [2018-01-25 21:22:55,864] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:22:55,866] - [rest_client:133] INFO - node 172.17.0.8 existing buckets : [] [2018-01-25 21:22:55,866] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. test_getl_expired_item (memcapable.GetlTests) ... ok ---------------------------------------------------------------------- Ran 1 test in 62.918s OK summary so far suite recreatebuckettests.RecreateMembaseBuckets , pass 0 , fail 1 failures so far... recreatebuckettests.RecreateMembaseBuckets.test_default_moxi summary so far suite deletebuckettests.DeleteMembaseBuckets , pass 1 , fail 0 summary so far suite createbuckettests.CreateMembaseBucketsTests , pass 1 , fail 4 failures so far... createbuckettests.CreateMembaseBucketsTests.test_default_moxi createbuckettests.CreateMembaseBucketsTests.test_default_on_non_default_port createbuckettests.CreateMembaseBucketsTests.test_non_default_case_sensitive_different_port createbuckettests.CreateMembaseBucketsTests.test_two_replica summary so far suite setgettests.MembaseBucket , pass 1 , fail 0 summary so far suite expirytests.ExpiryTests , pass 1 , fail 0 summary so far suite memcapable.GetlTests , pass 1 , fail 0 testrunner logs, diags and results are available under /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_10 Logs will be stored at /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_11 ./testrunner -i ./4node.ini -t memcapable.GetlTests.test_getl_thirty Test Input params: {'conf_file': './4node.conf', 'num_nodes': 4, 'cluster_name': '4node', 'ini': './4node.ini', 'case_number': 11, 'logs_folder': '/testrunner/logs/testrunner-18-Jan-25_21-16-28/test_11', 'spec': '4node'} [2018-01-25 21:22:57,891] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.8 [2018-01-25 21:22:57,892] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:23:19,994] - [rest_client:133] INFO - node 172.17.0.8 existing buckets : [] [2018-01-25 21:23:20,000] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise'] [2018-01-25 21:23:20,000] - [rest_client:2237] INFO - http://172.17.0.8:8091/pools/default/buckets with param: bucketType=membase&threadsNumber=3&authType=sasl&replicaIndex=1&name=default&saslPassword=&evictionPolicy=valueOnly&flushEnabled=1&replicaNumber=1&ramQuotaMB=4156 [2018-01-25 21:23:20,011] - [rest_client:2262] INFO - 0.01 seconds to create bucket default [2018-01-25 21:23:20,013] - [bucket_helper:343] INFO - waiting for memcached bucket : default in 172.17.0.8 to accept set ops [2018-01-25 21:23:21,148] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:23:21,283] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:23:22,518] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:23:22,582] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:23:22,773] - [rest_client:133] INFO - node 172.17.0.8 existing buckets : [u'default'] [2018-01-25 21:23:22,774] - [rest_client:137] INFO - node 172.17.0.8 found bucket default [2018-01-25 21:23:22,782] - [data_helper:295] INFO - creating direct client 172.17.0.8:11210 default [2018-01-25 21:23:22,811] - [memcapable:272] INFO - setting key getl_30_6847eb3f-3d93-4ecf-ac10-7de3e339c277 with expiration 0 [2018-01-25 21:23:22,812] - [memcapable:274] INFO - getl key getl_30_6847eb3f-3d93-4ecf-ac10-7de3e339c277 timeout 30 [2018-01-25 21:23:22,812] - [memcapable:282] INFO - get key getl_30_6847eb3f-3d93-4ecf-ac10-7de3e339c277 which is locked now [2018-01-25 21:23:22,813] - [memcapable:287] INFO - setting key getl_30_6847eb3f-3d93-4ecf-ac10-7de3e339c277 with new value * Memcached error #2 'Exists': for vbucket :405 to mc 172.17.0.8:11210 0 [2018-01-25 21:23:23,815] - [memcapable:287] INFO - setting key getl_30_6847eb3f-3d93-4ecf-ac10-7de3e339c277 with new value * Memcached error #2 'Exists': for vbucket :405 to mc 172.17.0.8:11210 1 [2018-01-25 21:23:24,818] - [memcapable:287] INFO - setting key getl_30_6847eb3f-3d93-4ecf-ac10-7de3e339c277 with new value * Memcached error #2 'Exists': for vbucket :405 to mc 172.17.0.8:11210 2 [2018-01-25 21:23:25,820] - [memcapable:287] INFO - setting key getl_30_6847eb3f-3d93-4ecf-ac10-7de3e339c277 with new value * Memcached error #2 'Exists': for vbucket :405 to mc 172.17.0.8:11210 3 [2018-01-25 21:23:26,823] - [memcapable:287] INFO - setting key getl_30_6847eb3f-3d93-4ecf-ac10-7de3e339c277 with new value * Memcached error #2 'Exists': for vbucket :405 to mc 172.17.0.8:11210 4 [2018-01-25 21:23:27,825] - [memcapable:287] INFO - setting key getl_30_6847eb3f-3d93-4ecf-ac10-7de3e339c277 with new value * Memcached error #2 'Exists': for vbucket :405 to mc 172.17.0.8:11210 5 [2018-01-25 21:23:28,826] - [memcapable:287] INFO - setting key getl_30_6847eb3f-3d93-4ecf-ac10-7de3e339c277 with new value * Memcached error #2 'Exists': for vbucket :405 to mc 172.17.0.8:11210 6 [2018-01-25 21:23:29,829] - [memcapable:287] INFO - setting key getl_30_6847eb3f-3d93-4ecf-ac10-7de3e339c277 with new value * Memcached error #2 'Exists': for vbucket :405 to mc 172.17.0.8:11210 7 [2018-01-25 21:23:30,831] - [memcapable:287] INFO - setting key getl_30_6847eb3f-3d93-4ecf-ac10-7de3e339c277 with new value * Memcached error #2 'Exists': for vbucket :405 to mc 172.17.0.8:11210 8 [2018-01-25 21:23:31,832] - [memcapable:287] INFO - setting key getl_30_6847eb3f-3d93-4ecf-ac10-7de3e339c277 with new value * Memcached error #2 'Exists': for vbucket :405 to mc 172.17.0.8:11210 9 [2018-01-25 21:23:32,834] - [memcapable:287] INFO - setting key getl_30_6847eb3f-3d93-4ecf-ac10-7de3e339c277 with new value * Memcached error #2 'Exists': for vbucket :405 to mc 172.17.0.8:11210 10 [2018-01-25 21:23:33,835] - [memcapable:287] INFO - setting key getl_30_6847eb3f-3d93-4ecf-ac10-7de3e339c277 with new value * Memcached error #2 'Exists': for vbucket :405 to mc 172.17.0.8:11210 11 [2018-01-25 21:23:34,837] - [memcapable:287] INFO - setting key getl_30_6847eb3f-3d93-4ecf-ac10-7de3e339c277 with new value * Memcached error #2 'Exists': for vbucket :405 to mc 172.17.0.8:11210 12 [2018-01-25 21:23:35,839] - [memcapable:287] INFO - setting key getl_30_6847eb3f-3d93-4ecf-ac10-7de3e339c277 with new value * Memcached error #2 'Exists': for vbucket :405 to mc 172.17.0.8:11210 13 [2018-01-25 21:23:36,841] - [memcapable:287] INFO - setting key getl_30_6847eb3f-3d93-4ecf-ac10-7de3e339c277 with new value * Memcached error #2 'Exists': for vbucket :405 to mc 172.17.0.8:11210 14 [2018-01-25 21:23:37,844] - [memcapable:287] INFO - setting key getl_30_6847eb3f-3d93-4ecf-ac10-7de3e339c277 with new value * Memcached error #2 'Exists': for vbucket :405 to mc 172.17.0.8:11210 15 [2018-01-25 21:23:38,846] - [memcapable:287] INFO - setting key getl_30_6847eb3f-3d93-4ecf-ac10-7de3e339c277 with new value * Memcached error #2 'Exists': for vbucket :405 to mc 172.17.0.8:11210 16 [2018-01-25 21:23:39,848] - [memcapable:287] INFO - setting key getl_30_6847eb3f-3d93-4ecf-ac10-7de3e339c277 with new value * Memcached error #2 'Exists': for vbucket :405 to mc 172.17.0.8:11210 17 [2018-01-25 21:23:40,850] - [memcapable:287] INFO - setting key getl_30_6847eb3f-3d93-4ecf-ac10-7de3e339c277 with new value * Memcached error #2 'Exists': for vbucket :405 to mc 172.17.0.8:11210 18 [2018-01-25 21:23:41,853] - [memcapable:287] INFO - setting key getl_30_6847eb3f-3d93-4ecf-ac10-7de3e339c277 with new value * Memcached error #2 'Exists': for vbucket :405 to mc 172.17.0.8:11210 19 [2018-01-25 21:23:42,854] - [memcapable:287] INFO - setting key getl_30_6847eb3f-3d93-4ecf-ac10-7de3e339c277 with new value * Memcached error #2 'Exists': for vbucket :405 to mc 172.17.0.8:11210 20 [2018-01-25 21:23:43,856] - [memcapable:287] INFO - setting key getl_30_6847eb3f-3d93-4ecf-ac10-7de3e339c277 with new value * Memcached error #2 'Exists': for vbucket :405 to mc 172.17.0.8:11210 21 [2018-01-25 21:23:44,859] - [memcapable:287] INFO - setting key getl_30_6847eb3f-3d93-4ecf-ac10-7de3e339c277 with new value * Memcached error #2 'Exists': for vbucket :405 to mc 172.17.0.8:11210 22 [2018-01-25 21:23:45,861] - [memcapable:287] INFO - setting key getl_30_6847eb3f-3d93-4ecf-ac10-7de3e339c277 with new value * Memcached error #2 'Exists': for vbucket :405 to mc 172.17.0.8:11210 23 [2018-01-25 21:23:46,862] - [memcapable:287] INFO - setting key getl_30_6847eb3f-3d93-4ecf-ac10-7de3e339c277 with new value * Memcached error #2 'Exists': for vbucket :405 to mc 172.17.0.8:11210 24 [2018-01-25 21:23:47,865] - [memcapable:287] INFO - setting key getl_30_6847eb3f-3d93-4ecf-ac10-7de3e339c277 with new value * Memcached error #2 'Exists': for vbucket :405 to mc 172.17.0.8:11210 25 [2018-01-25 21:23:48,883] - [memcapable:287] INFO - setting key getl_30_6847eb3f-3d93-4ecf-ac10-7de3e339c277 with new value * Memcached error #2 'Exists': for vbucket :405 to mc 172.17.0.8:11210 26 [2018-01-25 21:23:49,885] - [memcapable:287] INFO - setting key getl_30_6847eb3f-3d93-4ecf-ac10-7de3e339c277 with new value * Memcached error #2 'Exists': for vbucket :405 to mc 172.17.0.8:11210 27 [2018-01-25 21:23:50,886] - [memcapable:287] INFO - setting key getl_30_6847eb3f-3d93-4ecf-ac10-7de3e339c277 with new value * Memcached error #2 'Exists': for vbucket :405 to mc 172.17.0.8:11210 28 [2018-01-25 21:23:51,889] - [memcapable:287] INFO - setting key getl_30_6847eb3f-3d93-4ecf-ac10-7de3e339c277 with new value * Memcached error #2 'Exists': for vbucket :405 to mc 172.17.0.8:11210 29 [2018-01-25 21:23:52,892] - [memcapable:287] INFO - setting key getl_30_6847eb3f-3d93-4ecf-ac10-7de3e339c277 with new value * Memcached error #2 'Exists': for vbucket :405 to mc 172.17.0.8:11210 30 [2018-01-25 21:23:53,894] - [memcapable:287] INFO - setting key getl_30_6847eb3f-3d93-4ecf-ac10-7de3e339c277 with new value * [2018-01-25 21:23:53,895] - [memcapable:300] INFO - sleep for -15 seconds [2018-01-25 21:23:53,895] - [memcapable:301] INFO - sleep for 15 seconds [2018-01-25 21:24:23,909] - [memcapable:306] INFO - lock should have timed out by now . try to set the item again [2018-01-25 21:24:23,910] - [memcapable:308] INFO - setting key getl_30_6847eb3f-3d93-4ecf-ac10-7de3e339c277 with new value * [2018-01-25 21:24:23,910] - [memcapable:310] INFO - get key getl_30_6847eb3f-3d93-4ecf-ac10-7de3e339c277 [2018-01-25 21:24:23,940] - [bucket_helper:142] INFO - deleting existing buckets [u'default'] on 172.17.0.8 [2018-01-25 21:24:23,940] - [bucket_helper:144] INFO - remove bucket default ... [2018-01-25 21:24:24,313] - [bucket_helper:158] INFO - deleted bucket : default from 172.17.0.8 [2018-01-25 21:24:24,313] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:24:24,315] - [rest_client:133] INFO - node 172.17.0.8 existing buckets : [] [2018-01-25 21:24:24,315] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. test_getl_thirty (memcapable.GetlTests) ... ok ---------------------------------------------------------------------- Ran 1 test in 88.444s OK summary so far suite recreatebuckettests.RecreateMembaseBuckets , pass 0 , fail 1 failures so far... recreatebuckettests.RecreateMembaseBuckets.test_default_moxi summary so far suite deletebuckettests.DeleteMembaseBuckets , pass 1 , fail 0 summary so far suite createbuckettests.CreateMembaseBucketsTests , pass 1 , fail 4 failures so far... createbuckettests.CreateMembaseBucketsTests.test_default_moxi createbuckettests.CreateMembaseBucketsTests.test_default_on_non_default_port createbuckettests.CreateMembaseBucketsTests.test_non_default_case_sensitive_different_port createbuckettests.CreateMembaseBucketsTests.test_two_replica summary so far suite setgettests.MembaseBucket , pass 1 , fail 0 summary so far suite expirytests.ExpiryTests , pass 1 , fail 0 summary so far suite memcapable.GetlTests , pass 2 , fail 0 testrunner logs, diags and results are available under /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_11 Logs will be stored at /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_12 ./testrunner -i ./4node.ini -t memorysanitytests.MemorySanity.check_memory_stats,sasl_buckets=1,standard_buckets=1,items=2000 Test Input params: {'items': '2000', 'standard_buckets': '1', 'conf_file': './4node.conf', 'num_nodes': 4, 'cluster_name': '4node', 'ini': './4node.ini', 'case_number': 12, 'sasl_buckets': '1', 'logs_folder': '/testrunner/logs/testrunner-18-Jan-25_21-16-28/test_12', 'spec': '4node'} [2018-01-25 21:24:26,324] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 21:24:26,406] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 21:24:26,828] - [rest_client:1964] INFO - Node version in cluster 5.0.1-5003-enterprise [2018-01-25 21:24:26,837] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise'] [2018-01-25 21:24:26,837] - [basetestcase:201] INFO - ============== basetestcase setup was started for test #12 check_memory_stats============== [2018-01-25 21:24:26,848] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.8 [2018-01-25 21:24:26,848] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:24:28,857] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.6 [2018-01-25 21:24:28,857] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:24:30,866] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.7 [2018-01-25 21:24:30,866] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:24:32,875] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.5 [2018-01-25 21:24:32,875] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:24:34,893] - [basetestcase:482] INFO - sleep for 10 secs. ... [2018-01-25 21:24:44,906] - [cluster_helper:78] INFO - waiting for ns_server @ 172.17.0.8:8091 [2018-01-25 21:24:44,909] - [cluster_helper:80] INFO - ns_server @ 172.17.0.8:8091 is running [2018-01-25 21:24:44,913] - [cluster_helper:78] INFO - waiting for ns_server @ 172.17.0.6:8091 [2018-01-25 21:24:44,915] - [cluster_helper:80] INFO - ns_server @ 172.17.0.6:8091 is running [2018-01-25 21:24:44,918] - [cluster_helper:78] INFO - waiting for ns_server @ 172.17.0.7:8091 [2018-01-25 21:24:44,922] - [cluster_helper:80] INFO - ns_server @ 172.17.0.7:8091 is running [2018-01-25 21:24:44,925] - [cluster_helper:78] INFO - waiting for ns_server @ 172.17.0.5:8091 [2018-01-25 21:24:44,929] - [cluster_helper:80] INFO - ns_server @ 172.17.0.5:8091 is running [2018-01-25 21:24:44,929] - [basetestcase:245] INFO - initializing cluster [2018-01-25 21:24:45,352] - [task:125] INFO - server: ip:172.17.0.8 port:8091 ssh_username:root, nodes/self: {'ip': u'cb-example-0003.cb-example.default.svc', 'availableStorage': [], 'rest_username': '', 'id': u'ns_1@cb-example-0003.cb-example.default.svc', 'uptime': u'732', 'mcdMemoryReserved': 6234, 'storageTotalRam': 7793, 'hostname': u'cb-example-0003.cb-example.default.svc:8091', 'storage': [], 'moxi': 11211, 'port': u'8091', 'version': u'5.0.1-5003-enterprise', 'memcached': 11210, 'status': u'healthy', 'clusterCompatibility': 327680, 'curr_items': 0, 'services': [u'fts', u'index', u'kv', u'n1ql'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 5891821568, 'memoryTotal': 8171810816, 'memoryQuota': 6234, 'mcdMemoryAllocated': 6234, 'os': u'x86_64-unknown-linux-gnu', 'ports': []} [2018-01-25 21:24:45,352] - [task:157] INFO - quota for index service will be 256 MB [2018-01-25 21:24:45,353] - [task:159] INFO - set index quota to node 172.17.0.8 [2018-01-25 21:24:45,353] - [rest_client:919] INFO - pools/default params : indexMemoryQuota=256 [2018-01-25 21:24:45,358] - [rest_client:907] INFO - pools/default params : memoryQuota=3900 [2018-01-25 21:24:45,376] - [rest_client:942] INFO - settings/indexes params : storageMode=plasma [2018-01-25 21:24:45,384] - [rest_client:807] ERROR - POST http://172.17.0.8:8091/settings/indexes body: storageMode=plasma headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 400 reason: unknown {"errors":{"storageMode":"Changing the optimization mode of global indexes is not supported when index service nodes are present in the cluster. Please remove all index service nodes to change this option."}} auth: Administrator:password [2018-01-25 21:24:45,385] - [rest_client:885] INFO - /node/controller/setupServices params on 172.17.0.8: 8091:services=n1ql%2Ckv%2Cindex&password=password&hostname=172.17.0.8&user=Administrator [2018-01-25 21:24:45,386] - [rest_client:807] ERROR - POST http://172.17.0.8:8091//node/controller/setupServices body: services=n1ql%2Ckv%2Cindex&password=password&hostname=172.17.0.8&user=Administrator headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 400 reason: unknown ["cannot change node services after cluster is provisioned"] auth: Administrator:password [2018-01-25 21:24:45,386] - [rest_client:890] INFO - This node is already provisioned with services, we do not consider this as failure for test case [2018-01-25 21:24:45,387] - [rest_client:825] INFO - settings/web params on 172.17.0.8:8091:username=Administrator&password=password&port=8091 [2018-01-25 21:24:45,449] - [task:125] INFO - server: ip:172.17.0.6 port:8091 ssh_username:root, nodes/self: {'ip': u'cb-example-0001.cb-example.default.svc', 'availableStorage': [], 'rest_username': '', 'id': u'ns_1@cb-example-0001.cb-example.default.svc', 'uptime': u'759', 'mcdMemoryReserved': 6234, 'storageTotalRam': 7793, 'hostname': u'cb-example-0001.cb-example.default.svc:8091', 'storage': [], 'moxi': 11211, 'port': u'8091', 'version': u'5.0.1-5003-enterprise', 'memcached': 11210, 'status': u'healthy', 'clusterCompatibility': 327680, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 5890400256, 'memoryTotal': 8171810816, 'memoryQuota': 4675, 'mcdMemoryAllocated': 6234, 'os': u'x86_64-unknown-linux-gnu', 'ports': []} [2018-01-25 21:24:45,449] - [rest_client:907] INFO - pools/default params : memoryQuota=4156 [2018-01-25 21:24:45,455] - [rest_client:942] INFO - settings/indexes params : storageMode=plasma [2018-01-25 21:24:45,476] - [rest_client:825] INFO - settings/web params on 172.17.0.6:8091:username=Administrator&password=password&port=8091 [2018-01-25 21:24:45,531] - [task:125] INFO - server: ip:172.17.0.7 port:8091 ssh_username:root, nodes/self: {'ip': u'cb-example-0002.cb-example.default.svc', 'availableStorage': [], 'rest_username': '', 'id': u'ns_1@cb-example-0002.cb-example.default.svc', 'uptime': u'745', 'mcdMemoryReserved': 6234, 'storageTotalRam': 7793, 'hostname': u'cb-example-0002.cb-example.default.svc:8091', 'storage': [], 'moxi': 11211, 'port': u'8091', 'version': u'5.0.1-5003-enterprise', 'memcached': 11210, 'status': u'healthy', 'clusterCompatibility': 327680, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 5890367488, 'memoryTotal': 8171810816, 'memoryQuota': 4675, 'mcdMemoryAllocated': 6234, 'os': u'x86_64-unknown-linux-gnu', 'ports': []} [2018-01-25 21:24:45,531] - [rest_client:907] INFO - pools/default params : memoryQuota=4156 [2018-01-25 21:24:45,537] - [rest_client:942] INFO - settings/indexes params : storageMode=plasma [2018-01-25 21:24:45,545] - [rest_client:825] INFO - settings/web params on 172.17.0.7:8091:username=Administrator&password=password&port=8091 [2018-01-25 21:24:45,612] - [task:125] INFO - server: ip:172.17.0.5 port:8091 ssh_username:root, nodes/self: {'ip': u'cb-example-0000.cb-example.default.svc', 'availableStorage': [], 'rest_username': '', 'id': u'ns_1@cb-example-0000.cb-example.default.svc', 'uptime': u'782', 'mcdMemoryReserved': 6234, 'storageTotalRam': 7793, 'hostname': u'cb-example-0000.cb-example.default.svc:8091', 'storage': [], 'moxi': 11211, 'port': u'8091', 'version': u'5.0.1-5003-enterprise', 'memcached': 11210, 'status': u'healthy', 'clusterCompatibility': 327680, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 5890912256, 'memoryTotal': 8171810816, 'memoryQuota': 4675, 'mcdMemoryAllocated': 6234, 'os': u'x86_64-unknown-linux-gnu', 'ports': []} [2018-01-25 21:24:45,613] - [rest_client:907] INFO - pools/default params : memoryQuota=4156 [2018-01-25 21:24:45,617] - [rest_client:942] INFO - settings/indexes params : storageMode=plasma [2018-01-25 21:24:45,622] - [rest_client:825] INFO - settings/web params on 172.17.0.5:8091:username=Administrator&password=password&port=8091 [2018-01-25 21:24:45,717] - [basetestcase:2060] INFO - **** add built-in 'cbadminbucket' user to node 172.17.0.8 **** [2018-01-25 21:24:45,802] - [basetestcase:482] INFO - sleep for 10 secs. ... [2018-01-25 21:24:55,812] - [basetestcase:2065] INFO - **** add 'admin' role to 'cbadminbucket' user **** [2018-01-25 21:24:55,824] - [basetestcase:482] INFO - sleep for 10 secs. ... [2018-01-25 21:25:05,826] - [basetestcase:269] INFO - done initializing cluster [2018-01-25 21:25:06,725] - [task:425] INFO - adding node 172.17.0.6:8091 to cluster [2018-01-25 21:25:06,725] - [rest_client:1205] INFO - adding remote node @172.17.0.6:8091 to this cluster @172.17.0.8:8091 [2018-01-25 21:25:10,823] - [task:425] INFO - adding node 172.17.0.7:8091 to cluster [2018-01-25 21:25:10,823] - [rest_client:1205] INFO - adding remote node @172.17.0.7:8091 to this cluster @172.17.0.8:8091 [2018-01-25 21:25:14,592] - [task:425] INFO - adding node 172.17.0.5:8091 to cluster [2018-01-25 21:25:14,592] - [rest_client:1205] INFO - adding remote node @172.17.0.5:8091 to this cluster @172.17.0.8:8091 [2018-01-25 21:25:18,994] - [rest_client:1423] INFO - rebalance params : {'password': 'password', 'ejectedNodes': '', 'user': 'Administrator', 'knownNodes': u'ns_1@cb-example-0003.cb-example.default.svc,ns_1@172.17.0.7,ns_1@172.17.0.5,ns_1@172.17.0.6'} [2018-01-25 21:25:19,173] - [rest_client:1428] INFO - rebalance operation started [2018-01-25 21:25:19,191] - [rest_client:1576] INFO - rebalance percentage : 0.00 % [2018-01-25 21:25:19,191] - [task:491] INFO - Rebalance - status: running, progress: 0 [2018-01-25 21:25:29,351] - [task:491] INFO - Rebalance - status: none, progress: 100 [2018-01-25 21:25:29,356] - [task:550] INFO - rebalancing was completed with progress: 100% in 10.1827709675 sec [2018-01-25 21:25:30,378] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:25:30,378] - [rest_client:2237] INFO - http://172.17.0.8:8091/pools/default/buckets with param: bucketType=membase&threadsNumber=3&authType=sasl&replicaIndex=1&name=default&saslPassword=None&evictionPolicy=valueOnly&flushEnabled=1&replicaNumber=1&ramQuotaMB=1300 [2018-01-25 21:25:30,386] - [rest_client:2262] INFO - 0.01 seconds to create bucket default [2018-01-25 21:25:30,386] - [bucket_helper:343] INFO - waiting for memcached bucket : default in 172.17.0.8 to accept set ops [2018-01-25 21:25:33,146] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 default [2018-01-25 21:25:33,572] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 default [2018-01-25 21:25:33,898] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 default [2018-01-25 21:25:35,449] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:25:36,551] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 default [2018-01-25 21:25:36,880] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 default [2018-01-25 21:25:37,830] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 default [2018-01-25 21:25:38,369] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:25:38,766] - [task:312] INFO - bucket 'default' was created with per node RAM quota: 1300 [2018-01-25 21:25:40,048] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:25:40,049] - [rest_client:2237] INFO - http://172.17.0.8:8091/pools/default/buckets with param: bucketType=membase&threadsNumber=3&authType=sasl&replicaIndex=1&name=bucket0&saslPassword=password&evictionPolicy=valueOnly&flushEnabled=1&replicaNumber=1&ramQuotaMB=1300 [2018-01-25 21:25:40,073] - [rest_client:2262] INFO - 0.02 seconds to create bucket bucket0 [2018-01-25 21:25:40,073] - [bucket_helper:343] INFO - waiting for memcached bucket : bucket0 in 172.17.0.8 to accept set ops [2018-01-25 21:25:43,968] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 bucket0 [2018-01-25 21:25:44,447] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 bucket0 [2018-01-25 21:25:44,838] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 bucket0 [2018-01-25 21:25:45,287] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 bucket0 [2018-01-25 21:25:45,688] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 bucket0 [2018-01-25 21:25:45,986] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 bucket0 [2018-01-25 21:25:47,434] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 bucket0 [2018-01-25 21:25:48,140] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 bucket0 [2018-01-25 21:25:48,380] - [task:312] INFO - bucket 'bucket0' was created with per node RAM quota: 1300 [2018-01-25 21:25:49,403] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:25:49,403] - [rest_client:2237] INFO - http://172.17.0.8:8091/pools/default/buckets with param: bucketType=membase&threadsNumber=3&authType=none&replicaIndex=1&name=standard_bucket0&evictionPolicy=valueOnly&flushEnabled=1&replicaNumber=1&ramQuotaMB=1300 [2018-01-25 21:25:49,418] - [rest_client:2262] INFO - 0.01 seconds to create bucket standard_bucket0 [2018-01-25 21:25:49,418] - [bucket_helper:343] INFO - waiting for memcached bucket : standard_bucket0 in 172.17.0.8 to accept set ops [2018-01-25 21:25:51,922] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 standard_bucket0 [2018-01-25 21:25:52,302] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 standard_bucket0 [2018-01-25 21:25:52,825] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 standard_bucket0 [2018-01-25 21:25:53,196] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 standard_bucket0 [2018-01-25 21:25:55,821] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 standard_bucket0 [2018-01-25 21:25:56,506] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 standard_bucket0 [2018-01-25 21:25:56,635] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 standard_bucket0 [2018-01-25 21:25:56,709] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 standard_bucket0 [2018-01-25 21:25:57,141] - [task:312] INFO - bucket 'standard_bucket0' was created with per node RAM quota: 1300 [2018-01-25 21:25:57,163] - [basetestcase:344] INFO - ============== basetestcase setup was finished for test #12 check_memory_stats ============== [2018-01-25 21:25:57,180] - [basetestcase:482] INFO - sleep for 10 secs. ... [2018-01-25 21:26:07,189] - [memorysanitytests:19] INFO - ============== MemorySanityTest setup was started for test #12 check_memory_stats============== [2018-01-25 21:26:07,196] - [basetestcase:482] INFO - sleep for 5 secs. ... [2018-01-25 21:26:12,201] - [memorysanitytests:224] INFO - *** Check mem_used and mem_total before load data *** [2018-01-25 21:26:12,202] - [rest_client:1879] INFO - http://172.17.0.8:8091/pools/default/buckets/default/stats?zoom=minute [2018-01-25 21:26:12,228] - [rest_client:1879] INFO - http://172.17.0.8:8091/pools/default/buckets/default/stats?zoom=minute [2018-01-25 21:26:12,251] - [rest_client:1879] INFO - http://172.17.0.8:8091/pools/default/buckets/bucket0/stats?zoom=minute [2018-01-25 21:26:12,273] - [rest_client:1879] INFO - http://172.17.0.8:8091/pools/default/buckets/bucket0/stats?zoom=minute [2018-01-25 21:26:12,302] - [rest_client:1879] INFO - http://172.17.0.8:8091/pools/default/buckets/standard_bucket0/stats?zoom=minute [2018-01-25 21:26:12,324] - [rest_client:1879] INFO - http://172.17.0.8:8091/pools/default/buckets/standard_bucket0/stats?zoom=minute [2018-01-25 21:26:12,338] - [memorysanitytests:239] INFO - *** Load data to buckets *** [2018-01-25 21:26:12,403] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 default [2018-01-25 21:26:12,488] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 default [2018-01-25 21:26:12,564] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 default [2018-01-25 21:26:12,884] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:26:13,040] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 bucket0 [2018-01-25 21:26:13,111] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 bucket0 [2018-01-25 21:26:13,216] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 bucket0 [2018-01-25 21:26:13,326] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 default [2018-01-25 21:26:13,359] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 default [2018-01-25 21:26:13,418] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 default [2018-01-25 21:26:13,469] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 default [2018-01-25 21:26:13,521] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 bucket0 [2018-01-25 21:26:13,743] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 default [2018-01-25 21:26:13,792] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 default [2018-01-25 21:26:13,882] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 default [2018-01-25 21:26:13,940] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 default [2018-01-25 21:26:13,984] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 default [2018-01-25 21:26:13,988] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 default [2018-01-25 21:26:14,031] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 default [2018-01-25 21:26:14,104] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 standard_bucket0 [2018-01-25 21:26:14,134] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 default [2018-01-25 21:26:14,203] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:26:14,216] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:26:14,249] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:26:14,305] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:26:14,305] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 standard_bucket0 [2018-01-25 21:26:15,159] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 standard_bucket0 [2018-01-25 21:26:15,422] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 bucket0 [2018-01-25 21:26:15,868] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 bucket0 [2018-01-25 21:26:15,960] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 bucket0 [2018-01-25 21:26:15,964] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 standard_bucket0 [2018-01-25 21:26:15,993] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 bucket0 [2018-01-25 21:26:16,010] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 bucket0 [2018-01-25 21:26:16,169] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 bucket0 [2018-01-25 21:26:16,285] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 bucket0 [2018-01-25 21:26:16,299] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 bucket0 [2018-01-25 21:26:16,310] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 bucket0 [2018-01-25 21:26:16,329] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 bucket0 [2018-01-25 21:26:16,403] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 bucket0 [2018-01-25 21:26:16,409] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 bucket0 [2018-01-25 21:26:16,421] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 bucket0 [2018-01-25 21:26:17,422] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 bucket0 [2018-01-25 21:26:17,432] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 bucket0 [2018-01-25 21:26:17,457] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 bucket0 [2018-01-25 21:26:17,551] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 standard_bucket0 [2018-01-25 21:26:20,289] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 standard_bucket0 [2018-01-25 21:26:22,454] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 standard_bucket0 [2018-01-25 21:26:22,511] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 standard_bucket0 [2018-01-25 21:26:22,517] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 standard_bucket0 [2018-01-25 21:26:22,696] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 standard_bucket0 [2018-01-25 21:26:23,499] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 standard_bucket0 [2018-01-25 21:26:23,506] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 standard_bucket0 [2018-01-25 21:26:23,521] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 standard_bucket0 [2018-01-25 21:26:23,526] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 standard_bucket0 [2018-01-25 21:26:23,643] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 standard_bucket0 [2018-01-25 21:26:23,659] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 standard_bucket0 [2018-01-25 21:26:23,715] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 standard_bucket0 [2018-01-25 21:26:24,406] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 standard_bucket0 [2018-01-25 21:26:24,431] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 standard_bucket0 [2018-01-25 21:26:24,540] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 standard_bucket0 [2018-01-25 21:26:26,003] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:26:26,229] - [basetestcase:2242] INFO - list of all nodes in cluster: [ip:172.17.0.7 port:8091 ssh_username:root, ip:172.17.0.5 port:8091 ssh_username:root, ip:172.17.0.6 port:8091 ssh_username:root] [2018-01-25 21:26:27,568] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 default [2018-01-25 21:26:28,796] - [task:602] INFO - Saw ep_queue_size 0 == 0 expected on '172.17.0.6:8091',default bucket [2018-01-25 21:26:28,814] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 bucket0 [2018-01-25 21:26:28,884] - [task:602] INFO - Saw ep_queue_size 0 == 0 expected on '172.17.0.6:8091',bucket0 bucket [2018-01-25 21:26:28,897] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 standard_bucket0 [2018-01-25 21:26:28,973] - [task:602] INFO - Saw ep_queue_size 0 == 0 expected on '172.17.0.6:8091',standard_bucket0 bucket [2018-01-25 21:26:28,988] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 default [2018-01-25 21:26:29,068] - [task:602] INFO - Saw ep_queue_size 0 == 0 expected on '172.17.0.7:8091',default bucket [2018-01-25 21:26:29,082] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 bucket0 [2018-01-25 21:26:29,142] - [task:602] INFO - Saw ep_queue_size 0 == 0 expected on '172.17.0.7:8091',bucket0 bucket [2018-01-25 21:26:29,152] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 standard_bucket0 [2018-01-25 21:26:29,202] - [task:602] INFO - Saw ep_queue_size 0 == 0 expected on '172.17.0.7:8091',standard_bucket0 bucket [2018-01-25 21:26:29,217] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 default [2018-01-25 21:26:29,292] - [task:602] INFO - Saw ep_queue_size 0 == 0 expected on '172.17.0.5:8091',default bucket [2018-01-25 21:26:29,305] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 bucket0 [2018-01-25 21:26:29,424] - [task:602] INFO - Saw ep_queue_size 0 == 0 expected on '172.17.0.5:8091',bucket0 bucket [2018-01-25 21:26:29,438] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 standard_bucket0 [2018-01-25 21:26:29,498] - [task:602] INFO - Saw ep_queue_size 0 == 0 expected on '172.17.0.5:8091',standard_bucket0 bucket [2018-01-25 21:26:29,529] - [memorysanitytests:244] INFO - *** Check mem_used and mem_total after load data *** [2018-01-25 21:26:29,529] - [rest_client:1879] INFO - http://172.17.0.8:8091/pools/default/buckets/default/stats?zoom=minute [2018-01-25 21:26:29,571] - [rest_client:1879] INFO - http://172.17.0.8:8091/pools/default/buckets/default/stats?zoom=minute [2018-01-25 21:26:29,650] - [memorysanitytests:267] INFO - Mem stats works as expected [2018-01-25 21:26:29,650] - [memorysanitytests:269] INFO - Initial memory used 48952856 [2018-01-25 21:26:29,650] - [memorysanitytests:271] INFO - Memory after loaded 51917448 [2018-01-25 21:26:29,650] - [rest_client:1879] INFO - http://172.17.0.8:8091/pools/default/buckets/bucket0/stats?zoom=minute [2018-01-25 21:26:29,689] - [rest_client:1879] INFO - http://172.17.0.8:8091/pools/default/buckets/bucket0/stats?zoom=minute [2018-01-25 21:26:29,726] - [memorysanitytests:267] INFO - Mem stats works as expected [2018-01-25 21:26:29,726] - [memorysanitytests:269] INFO - Initial memory used 48953576 [2018-01-25 21:26:29,726] - [memorysanitytests:271] INFO - Memory after loaded 51895592 [2018-01-25 21:26:29,726] - [rest_client:1879] INFO - http://172.17.0.8:8091/pools/default/buckets/standard_bucket0/stats?zoom=minute [2018-01-25 21:26:29,753] - [rest_client:1879] INFO - http://172.17.0.8:8091/pools/default/buckets/standard_bucket0/stats?zoom=minute [2018-01-25 21:26:29,826] - [memorysanitytests:267] INFO - Mem stats works as expected [2018-01-25 21:26:29,827] - [memorysanitytests:269] INFO - Initial memory used 48954480 [2018-01-25 21:26:29,828] - [memorysanitytests:271] INFO - Memory after loaded 52789736 [2018-01-25 21:26:29,828] - [memorysanitytests:24] INFO - ============== teardown was started for test #12 check_memory_stats============== [2018-01-25 21:26:29,828] - [basetestcase:414] INFO - ============== basetestcase cleanup was started for test #12 check_memory_stats ============== [2018-01-25 21:26:29,979] - [bucket_helper:142] INFO - deleting existing buckets [u'bucket0', u'default', u'standard_bucket0'] on 172.17.0.8 [2018-01-25 21:26:29,981] - [bucket_helper:144] INFO - remove bucket bucket0 ... [2018-01-25 21:26:31,486] - [bucket_helper:158] INFO - deleted bucket : bucket0 from 172.17.0.8 [2018-01-25 21:26:31,486] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:26:31,582] - [rest_client:133] INFO - node 172.17.0.8 existing buckets : [u'default', u'standard_bucket0'] [2018-01-25 21:26:31,583] - [bucket_helper:144] INFO - remove bucket default ... [2018-01-25 21:26:33,721] - [bucket_helper:158] INFO - deleted bucket : default from 172.17.0.8 [2018-01-25 21:26:33,721] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:26:33,736] - [rest_client:133] INFO - node 172.17.0.8 existing buckets : [u'standard_bucket0'] [2018-01-25 21:26:33,736] - [bucket_helper:144] INFO - remove bucket standard_bucket0 ... [2018-01-25 21:26:34,840] - [bucket_helper:158] INFO - deleted bucket : standard_bucket0 from 172.17.0.8 [2018-01-25 21:26:34,840] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:26:34,847] - [rest_client:133] INFO - node 172.17.0.8 existing buckets : [] [2018-01-25 21:26:34,848] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:26:36,875] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.6 [2018-01-25 21:26:36,875] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:26:38,885] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.7 [2018-01-25 21:26:38,885] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:26:40,895] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.5 [2018-01-25 21:26:40,895] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:26:42,922] - [cluster_helper:254] INFO - rebalancing all nodes in order to remove nodes [2018-01-25 21:26:42,928] - [rest_client:1423] INFO - rebalance params : {'password': 'password', 'ejectedNodes': u'ns_1@172.17.0.7,ns_1@172.17.0.5,ns_1@172.17.0.6', 'user': 'Administrator', 'knownNodes': u'ns_1@cb-example-0003.cb-example.default.svc,ns_1@172.17.0.7,ns_1@172.17.0.5,ns_1@172.17.0.6'} [2018-01-25 21:26:42,943] - [rest_client:1428] INFO - rebalance operation started [2018-01-25 21:26:42,951] - [rest_client:1576] INFO - rebalance percentage : 0.00 % [2018-01-25 21:26:52,963] - [rest_client:1576] INFO - rebalance percentage : 58.33 % [2018-01-25 21:27:02,975] - [rest_client:1576] INFO - rebalance percentage : 58.33 % [2018-01-25 21:27:22,999] - [rest_client:1515] INFO - rebalance progress took 40.0559999943 seconds [2018-01-25 21:27:22,999] - [rest_client:1516] INFO - sleep for 10 seconds after rebalance... [2018-01-25 21:27:33,043] - [cluster_helper:294] INFO - removed all the nodes from cluster associated with ip:172.17.0.8 port:8091 ssh_username:root ? [(u'ns_1@172.17.0.7', 8091), (u'ns_1@172.17.0.5', 8091), (u'ns_1@172.17.0.6', 8091)] [2018-01-25 21:27:33,051] - [cluster_helper:78] INFO - waiting for ns_server @ 172.17.0.8:8091 [2018-01-25 21:27:33,056] - [cluster_helper:80] INFO - ns_server @ 172.17.0.8:8091 is running [2018-01-25 21:27:33,061] - [cluster_helper:78] INFO - waiting for ns_server @ 172.17.0.6:8091 [2018-01-25 21:27:33,065] - [cluster_helper:80] INFO - ns_server @ 172.17.0.6:8091 is running [2018-01-25 21:27:33,068] - [cluster_helper:78] INFO - waiting for ns_server @ 172.17.0.7:8091 [2018-01-25 21:27:33,072] - [cluster_helper:80] INFO - ns_server @ 172.17.0.7:8091 is running [2018-01-25 21:27:33,076] - [cluster_helper:78] INFO - waiting for ns_server @ 172.17.0.5:8091 [2018-01-25 21:27:33,103] - [cluster_helper:80] INFO - ns_server @ 172.17.0.5:8091 is running [2018-01-25 21:27:33,104] - [basetestcase:432] INFO - ============== basetestcase cleanup was finished for test #12 check_memory_stats ============== check_memory_stats (memorysanitytests.MemorySanity) ... ok ---------------------------------------------------------------------- Ran 1 test in 186.794s OK Cluster instance shutdown with force summary so far suite recreatebuckettests.RecreateMembaseBuckets , pass 0 , fail 1 failures so far... recreatebuckettests.RecreateMembaseBuckets.test_default_moxi summary so far suite deletebuckettests.DeleteMembaseBuckets , pass 1 , fail 0 summary so far suite createbuckettests.CreateMembaseBucketsTests , pass 1 , fail 4 failures so far... createbuckettests.CreateMembaseBucketsTests.test_default_moxi createbuckettests.CreateMembaseBucketsTests.test_default_on_non_default_port createbuckettests.CreateMembaseBucketsTests.test_non_default_case_sensitive_different_port createbuckettests.CreateMembaseBucketsTests.test_two_replica summary so far suite setgettests.MembaseBucket , pass 1 , fail 0 summary so far suite expirytests.ExpiryTests , pass 1 , fail 0 summary so far suite memcapable.GetlTests , pass 2 , fail 0 summary so far suite memorysanitytests.MemorySanity , pass 1 , fail 0 testrunner logs, diags and results are available under /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_12 Logs will be stored at /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_13 ./testrunner -i ./4node.ini -t drainratetests.DrainRateTests.test_drain_100k_items Test Input params: {'conf_file': './4node.conf', 'num_nodes': 4, 'cluster_name': '4node', 'ini': './4node.ini', 'case_number': 13, 'logs_folder': '/testrunner/logs/testrunner-18-Jan-25_21-16-28/test_13', 'spec': '4node'} [2018-01-25 21:27:53,281] - [rest_client:133] INFO - node 172.17.0.8 existing buckets : [] [2018-01-25 21:27:53,301] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise'] [2018-01-25 21:27:53,301] - [rest_client:2237] INFO - http://172.17.0.8:8091/pools/default/buckets with param: bucketType=membase&threadsNumber=3&authType=sasl&replicaIndex=1&name=default&saslPassword=&evictionPolicy=valueOnly&flushEnabled=1&replicaNumber=1&ramQuotaMB=2600 [2018-01-25 21:27:53,332] - [rest_client:2262] INFO - 0.03 seconds to create bucket default [2018-01-25 21:27:53,335] - [bucket_helper:343] INFO - waiting for memcached bucket : default in 172.17.0.8 to accept set ops [2018-01-25 21:27:54,922] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:27:55,423] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:27:57,927] - [rest_client:133] INFO - node 172.17.0.8 existing buckets : [u'default'] [2018-01-25 21:27:58,025] - [rest_client:137] INFO - node 172.17.0.8 found bucket default [2018-01-25 21:27:58,026] - [drainratetests:124] INFO - waiting for loader thread to insert 100000 items [2018-01-25 21:27:58,053] - [data_helper:96] INFO - each thread will send 100000.0 items with value of size : 128 [2018-01-25 21:27:58,065] - [data_helper:563] INFO - starting a thread to set keys mixed set-get ? True and using async_set ? False with moxi ? True [2018-01-25 21:27:58,066] - [data_helper:206] INFO - waiting for all worker thread to finish their work... [2018-01-25 21:27:58,142] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:27:58,215] - [data_helper:357] INFO - creating proxy client 172.17.0.8:11211 default [2018-01-25 21:27:59,032] - [data_helper:570] INFO - unable to create memcached client due to [Errno 32] Broken pipe. stop thread... test_drain_100k_items (drainratetests.DrainRateTests) ... Traceback (most recent call last): File "lib/memcached/helper/data_helper.py", line 568, in run client = MemcachedClientHelper.proxy_client(self.serverInfo, self.name) File "lib/memcached/helper/data_helper.py", line 362, in proxy_client bucket_info.saslPassword.encode('ascii')) File "lib/mc_bin_client.py", line 529, in sasl_auth_plain return self.sasl_auth_start('PLAIN', '\0'.join([foruser, user, password])) File "lib/mc_bin_client.py", line 525, in sasl_auth_start return self._doCmd(memcacheConstants.CMD_SASL_AUTH, mech, data) File "lib/mc_bin_client.py", line 146, in _doCmd extraHeaderLength=extraHeaderLength) File "lib/mc_bin_client.py", line 76, in _sendCmd vbucketId=self.vbucketId, extended_meta_data=extended_meta_data, extraHeaderLength=extraHeaderLength) File "lib/mc_bin_client.py", line 92, in _sendMsg self.s.send(msg + extraHeader + key + val + extended_meta_data) error: [Errno 32] Broken pipe [2018-01-25 21:27:59,088] - [data_helper:208] INFO - worker threads are done... [2018-01-25 21:27:59,340] - [data_helper:223] INFO - inserted keys count : 0 , rejected keys count : 0 [2018-01-25 21:27:59,996] - [data_helper:225] INFO - deleted keys count : 0 , expired keys count : 0 [2018-01-25 21:28:00,002] - [drainratetests:131] INFO - waiting for ep_queue == 0 [2018-01-25 21:28:00,008] - [drainratetests:112] INFO - wait 2 seconds for bucket stats are up [2018-01-25 21:28:02,019] - [drainratetests:115] INFO - current ep_queue_size: 0 [2018-01-25 21:28:02,019] - [rebalance_helper:22] INFO - waiting for bucket default stat : ep_queue_size to match 0 on 172.17.0.8 [2018-01-25 21:28:02,075] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'cb-example-0003.cb-example.default.svc', 'password': 'password', 'port': 8091} [2018-01-25 21:28:02,075] - [data_helper:293] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:28:02,221] - [rebalance_helper:46] INFO - 0 [2018-01-25 21:28:02,223] - [rebalance_helper:51] INFO - ep_queue_size : 0 [2018-01-25 21:28:02,223] - [rebalance_helper:22] INFO - waiting for bucket default stat : ep_flusher_todo to match 0 on 172.17.0.8 [2018-01-25 21:28:02,328] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'cb-example-0003.cb-example.default.svc', 'password': 'password', 'port': 8091} [2018-01-25 21:28:02,328] - [data_helper:293] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:28:02,433] - [rebalance_helper:46] INFO - 0 [2018-01-25 21:28:02,434] - [rebalance_helper:51] INFO - ep_flusher_todo : 0 [2018-01-25 21:28:02,434] - [rebalance_helper:22] INFO - waiting for bucket default stat : ep_uncommitted_items to match 0 on 172.17.0.8 [2018-01-25 21:28:02,487] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'cb-example-0003.cb-example.default.svc', 'password': 'password', 'port': 8091} [2018-01-25 21:28:02,487] - [data_helper:293] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:28:02,529] - [rebalance_helper:46] INFO - 0 [2018-01-25 21:28:02,530] - [rebalance_helper:51] INFO - ep_uncommitted_items : 0 [2018-01-25 21:28:02,530] - [drainratetests:133] INFO - took 2.52144002914 seconds to drain 100000 items [2018-01-25 21:28:02,543] - [bucket_helper:142] INFO - deleting existing buckets [u'default'] on 172.17.0.8 [2018-01-25 21:28:02,543] - [bucket_helper:144] INFO - remove bucket default ... [2018-01-25 21:28:04,059] - [bucket_helper:158] INFO - deleted bucket : default from 172.17.0.8 [2018-01-25 21:28:04,060] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:28:04,076] - [rest_client:133] INFO - node 172.17.0.8 existing buckets : [] [2018-01-25 21:28:04,077] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. ok ---------------------------------------------------------------------- Ran 1 test in 32.989s OK summary so far suite recreatebuckettests.RecreateMembaseBuckets , pass 0 , fail 1 failures so far... recreatebuckettests.RecreateMembaseBuckets.test_default_moxi summary so far suite deletebuckettests.DeleteMembaseBuckets , pass 1 , fail 0 summary so far suite createbuckettests.CreateMembaseBucketsTests , pass 1 , fail 4 failures so far... createbuckettests.CreateMembaseBucketsTests.test_default_moxi createbuckettests.CreateMembaseBucketsTests.test_default_on_non_default_port createbuckettests.CreateMembaseBucketsTests.test_non_default_case_sensitive_different_port createbuckettests.CreateMembaseBucketsTests.test_two_replica summary so far suite setgettests.MembaseBucket , pass 1 , fail 0 summary so far suite expirytests.ExpiryTests , pass 1 , fail 0 summary so far suite memcapable.GetlTests , pass 2 , fail 0 summary so far suite memorysanitytests.MemorySanity , pass 1 , fail 0 summary so far suite drainratetests.DrainRateTests , pass 1 , fail 0 testrunner logs, diags and results are available under /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_13 Logs will be stored at /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_14 ./testrunner -i ./4node.ini -t view.viewquerytests.ViewQueryTests.test_employee_dataset_all_queries,limit=1000,docs-per-day=2,wait_persistence=true,timeout=1200 Test Input params: {'docs-per-day': '2', 'wait_persistence': 'true', 'conf_file': './4node.conf', 'num_nodes': 4, 'cluster_name': '4node', 'limit': '1000', 'ini': './4node.ini', 'timeout': '1200', 'case_number': 14, 'logs_folder': '/testrunner/logs/testrunner-18-Jan-25_21-16-28/test_14', 'spec': '4node'} [2018-01-25 21:28:06,238] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 21:28:06,374] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 21:28:06,857] - [rest_client:1964] INFO - Node version in cluster 5.0.1-5003-enterprise [2018-01-25 21:28:06,863] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise'] [2018-01-25 21:28:06,863] - [basetestcase:201] INFO - ============== basetestcase setup was started for test #14 test_employee_dataset_all_queries============== [2018-01-25 21:28:06,880] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.8 [2018-01-25 21:28:06,880] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:28:08,904] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.6 [2018-01-25 21:28:08,904] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:28:11,032] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.7 [2018-01-25 21:28:11,033] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:28:13,049] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.5 [2018-01-25 21:28:13,050] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:28:15,076] - [basetestcase:482] INFO - sleep for 10 secs. ... [2018-01-25 21:28:25,094] - [cluster_helper:78] INFO - waiting for ns_server @ 172.17.0.8:8091 [2018-01-25 21:28:25,105] - [cluster_helper:80] INFO - ns_server @ 172.17.0.8:8091 is running [2018-01-25 21:28:25,116] - [cluster_helper:78] INFO - waiting for ns_server @ 172.17.0.6:8091 [2018-01-25 21:28:25,126] - [cluster_helper:80] INFO - ns_server @ 172.17.0.6:8091 is running [2018-01-25 21:28:25,131] - [cluster_helper:78] INFO - waiting for ns_server @ 172.17.0.7:8091 [2018-01-25 21:28:25,136] - [cluster_helper:80] INFO - ns_server @ 172.17.0.7:8091 is running [2018-01-25 21:28:25,139] - [cluster_helper:78] INFO - waiting for ns_server @ 172.17.0.5:8091 [2018-01-25 21:28:25,147] - [cluster_helper:80] INFO - ns_server @ 172.17.0.5:8091 is running [2018-01-25 21:28:25,148] - [basetestcase:245] INFO - initializing cluster [2018-01-25 21:28:25,311] - [task:125] INFO - server: ip:172.17.0.8 port:8091 ssh_username:root, nodes/self: {'ip': u'cb-example-0003.cb-example.default.svc', 'availableStorage': [], 'rest_username': '', 'id': u'ns_1@cb-example-0003.cb-example.default.svc', 'uptime': u'952', 'mcdMemoryReserved': 6234, 'storageTotalRam': 7793, 'hostname': u'cb-example-0003.cb-example.default.svc:8091', 'storage': [], 'moxi': 11211, 'port': u'8091', 'version': u'5.0.1-5003-enterprise', 'memcached': 11210, 'status': u'healthy', 'clusterCompatibility': 327680, 'curr_items': 0, 'services': [u'fts', u'index', u'kv', u'n1ql'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 5722914816, 'memoryTotal': 8171810816, 'memoryQuota': 3900, 'mcdMemoryAllocated': 6234, 'os': u'x86_64-unknown-linux-gnu', 'ports': []} [2018-01-25 21:28:25,312] - [task:157] INFO - quota for index service will be 256 MB [2018-01-25 21:28:25,312] - [task:159] INFO - set index quota to node 172.17.0.8 [2018-01-25 21:28:25,312] - [rest_client:919] INFO - pools/default params : indexMemoryQuota=256 [2018-01-25 21:28:25,325] - [rest_client:907] INFO - pools/default params : memoryQuota=3900 [2018-01-25 21:28:25,391] - [rest_client:942] INFO - settings/indexes params : storageMode=plasma [2018-01-25 21:28:25,396] - [rest_client:807] ERROR - POST http://172.17.0.8:8091/settings/indexes body: storageMode=plasma headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 400 reason: unknown {"errors":{"storageMode":"Changing the optimization mode of global indexes is not supported when index service nodes are present in the cluster. Please remove all index service nodes to change this option."}} auth: Administrator:password [2018-01-25 21:28:25,399] - [rest_client:885] INFO - /node/controller/setupServices params on 172.17.0.8: 8091:services=n1ql%2Ckv%2Cindex&password=password&hostname=172.17.0.8&user=Administrator [2018-01-25 21:28:29,267] - [rest_client:807] ERROR - POST http://172.17.0.8:8091//node/controller/setupServices body: services=n1ql%2Ckv%2Cindex&password=password&hostname=172.17.0.8&user=Administrator headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 400 reason: unknown ["cannot change node services after cluster is provisioned"] auth: Administrator:password [2018-01-25 21:28:29,272] - [rest_client:890] INFO - This node is already provisioned with services, we do not consider this as failure for test case [2018-01-25 21:28:29,283] - [rest_client:825] INFO - settings/web params on 172.17.0.8:8091:username=Administrator&password=password&port=8091 [2018-01-25 21:28:47,288] - [task:125] INFO - server: ip:172.17.0.6 port:8091 ssh_username:root, nodes/self: {'ip': u'172.17.0.6', 'availableStorage': [], 'rest_username': '', 'id': u'ns_1@172.17.0.6', 'uptime': u'1003', 'mcdMemoryReserved': 6234, 'storageTotalRam': 7793, 'hostname': u'172.17.0.6:8091', 'storage': [], 'moxi': 11211, 'port': u'8091', 'version': u'5.0.1-5003-enterprise', 'memcached': 11210, 'status': u'healthy', 'clusterCompatibility': 327680, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 5714329600, 'memoryTotal': 8171810816, 'memoryQuota': 4675, 'mcdMemoryAllocated': 6234, 'os': u'x86_64-unknown-linux-gnu', 'ports': []} [2018-01-25 21:28:47,288] - [rest_client:907] INFO - pools/default params : memoryQuota=4156 [2018-01-25 21:28:47,309] - [rest_client:942] INFO - settings/indexes params : storageMode=plasma [2018-01-25 21:28:47,332] - [rest_client:825] INFO - settings/web params on 172.17.0.6:8091:username=Administrator&password=password&port=8091 [2018-01-25 21:28:47,432] - [task:125] INFO - server: ip:172.17.0.7 port:8091 ssh_username:root, nodes/self: {'ip': u'172.17.0.7', 'availableStorage': [], 'rest_username': '', 'id': u'ns_1@172.17.0.7', 'uptime': u'988', 'mcdMemoryReserved': 6234, 'storageTotalRam': 7793, 'hostname': u'172.17.0.7:8091', 'storage': [], 'moxi': 11211, 'port': u'8091', 'version': u'5.0.1-5003-enterprise', 'memcached': 11210, 'status': u'healthy', 'clusterCompatibility': 327680, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 5714280448, 'memoryTotal': 8171810816, 'memoryQuota': 4675, 'mcdMemoryAllocated': 6234, 'os': u'x86_64-unknown-linux-gnu', 'ports': []} [2018-01-25 21:28:47,433] - [rest_client:907] INFO - pools/default params : memoryQuota=4156 [2018-01-25 21:28:47,437] - [rest_client:942] INFO - settings/indexes params : storageMode=plasma [2018-01-25 21:28:47,444] - [rest_client:825] INFO - settings/web params on 172.17.0.7:8091:username=Administrator&password=password&port=8091 [2018-01-25 21:28:47,630] - [task:125] INFO - server: ip:172.17.0.5 port:8091 ssh_username:root, nodes/self: {'ip': u'172.17.0.5', 'availableStorage': [], 'rest_username': '', 'id': u'ns_1@172.17.0.5', 'uptime': u'1025', 'mcdMemoryReserved': 6234, 'storageTotalRam': 7793, 'hostname': u'172.17.0.5:8091', 'storage': [], 'moxi': 11211, 'port': u'8091', 'version': u'5.0.1-5003-enterprise', 'memcached': 11210, 'status': u'healthy', 'clusterCompatibility': 327680, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 5713932288, 'memoryTotal': 8171810816, 'memoryQuota': 4675, 'mcdMemoryAllocated': 6234, 'os': u'x86_64-unknown-linux-gnu', 'ports': []} [2018-01-25 21:28:47,630] - [rest_client:907] INFO - pools/default params : memoryQuota=4156 [2018-01-25 21:28:47,635] - [rest_client:942] INFO - settings/indexes params : storageMode=plasma [2018-01-25 21:28:47,641] - [rest_client:825] INFO - settings/web params on 172.17.0.5:8091:username=Administrator&password=password&port=8091 [2018-01-25 21:28:47,701] - [basetestcase:2060] INFO - **** add built-in 'cbadminbucket' user to node 172.17.0.8 **** [2018-01-25 21:28:47,720] - [rest_client:807] ERROR - DELETE http://172.17.0.8:8091/settings/rbac/users/local/cbadminbucket body: headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 404 reason: unknown "User was not found." auth: Administrator:password [2018-01-25 21:28:47,726] - [internal_user:36] INFO - Exception while deleting user. Exception is -"User was not found." [2018-01-25 21:28:47,784] - [basetestcase:482] INFO - sleep for 10 secs. ... [2018-01-25 21:28:57,788] - [basetestcase:2065] INFO - **** add 'admin' role to 'cbadminbucket' user **** [2018-01-25 21:28:57,808] - [basetestcase:482] INFO - sleep for 10 secs. ... [2018-01-25 21:29:07,987] - [basetestcase:269] INFO - done initializing cluster [2018-01-25 21:29:09,033] - [task:425] INFO - adding node 172.17.0.6:8091 to cluster [2018-01-25 21:29:09,033] - [rest_client:1205] INFO - adding remote node @172.17.0.6:8091 to this cluster @172.17.0.8:8091 [2018-01-25 21:29:15,170] - [task:425] INFO - adding node 172.17.0.7:8091 to cluster [2018-01-25 21:29:15,171] - [rest_client:1205] INFO - adding remote node @172.17.0.7:8091 to this cluster @172.17.0.8:8091 [2018-01-25 21:29:22,006] - [task:425] INFO - adding node 172.17.0.5:8091 to cluster [2018-01-25 21:29:22,006] - [rest_client:1205] INFO - adding remote node @172.17.0.5:8091 to this cluster @172.17.0.8:8091 [2018-01-25 21:29:27,157] - [rest_client:1423] INFO - rebalance params : {'password': 'password', 'ejectedNodes': '', 'user': 'Administrator', 'knownNodes': u'ns_1@cb-example-0003.cb-example.default.svc,ns_1@172.17.0.7,ns_1@172.17.0.5,ns_1@172.17.0.6'} [2018-01-25 21:29:27,164] - [rest_client:1428] INFO - rebalance operation started [2018-01-25 21:29:27,196] - [rest_client:1576] INFO - rebalance percentage : 0.00 % [2018-01-25 21:29:27,196] - [task:491] INFO - Rebalance - status: running, progress: 0 [2018-01-25 21:29:37,575] - [task:491] INFO - Rebalance - status: none, progress: 100 [2018-01-25 21:29:37,582] - [task:550] INFO - rebalancing was completed with progress: 100% in 10.4174571037 sec [2018-01-25 21:29:38,607] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:29:38,607] - [rest_client:2237] INFO - http://172.17.0.8:8091/pools/default/buckets with param: bucketType=membase&threadsNumber=3&authType=sasl&replicaIndex=1&name=default&saslPassword=None&evictionPolicy=valueOnly&flushEnabled=1&replicaNumber=1&ramQuotaMB=3900 [2018-01-25 21:29:38,623] - [rest_client:2262] INFO - 0.02 seconds to create bucket default [2018-01-25 21:29:38,624] - [bucket_helper:343] INFO - waiting for memcached bucket : default in 172.17.0.8 to accept set ops [2018-01-25 21:29:45,159] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 default [2018-01-25 21:29:45,722] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 default [2018-01-25 21:29:46,203] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 default [2018-01-25 21:29:46,755] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:29:47,595] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 default [2018-01-25 21:29:48,521] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 default [2018-01-25 21:29:48,632] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 default [2018-01-25 21:29:48,687] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:29:48,903] - [task:312] INFO - bucket 'default' was created with per node RAM quota: 3900 [2018-01-25 21:29:48,930] - [basetestcase:344] INFO - ============== basetestcase setup was finished for test #14 test_employee_dataset_all_queries ============== [2018-01-25 21:29:48,948] - [basetestcase:482] INFO - sleep for 10 secs. ... [2018-01-25 21:29:59,073] - [rest_client:807] ERROR - GET http://cb-example-0003.cb-example.default.svc:8092//default/_design/test_view-4e17ebd body: headers: {'Content-Type': 'application/json', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 404 reason: not_found {"error":"not_found","reason":"missing"} auth: Administrator:password [2018-01-25 21:29:59,140] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-4e17ebd/_view/test_view-4e17ebd?stale=ok [2018-01-25 21:29:59,664] - [task:2181] INFO - view : test_view-4e17ebd was created successfully in ddoc: test_view-4e17ebd [2018-01-25 21:30:00,977] - [rest_client:807] ERROR - GET http://cb-example-0003.cb-example.default.svc:8092//default/_design/test_view-52950e3 body: headers: {'Content-Type': 'application/json', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 404 reason: not_found {"error":"not_found","reason":"missing"} auth: Administrator:password [2018-01-25 21:30:01,055] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-52950e3/_view/test_view-52950e3?stale=ok [2018-01-25 21:30:01,184] - [task:2181] INFO - view : test_view-52950e3 was created successfully in ddoc: test_view-52950e3 [2018-01-25 21:30:02,217] - [rest_client:807] ERROR - GET http://cb-example-0003.cb-example.default.svc:8092//default/_design/test_view-edcbe72 body: headers: {'Content-Type': 'application/json', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 404 reason: not_found {"error":"not_found","reason":"missing"} auth: Administrator:password [2018-01-25 21:30:02,282] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-edcbe72/_view/test_view-edcbe72?stale=ok [2018-01-25 21:30:07,457] - [task:2181] INFO - view : test_view-edcbe72 was created successfully in ddoc: test_view-edcbe72 [2018-01-25 21:30:08,513] - [rest_client:807] ERROR - GET http://cb-example-0003.cb-example.default.svc:8092//default/_design/test_view-3aef3b0 body: headers: {'Content-Type': 'application/json', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 404 reason: not_found {"error":"not_found","reason":"missing"} auth: Administrator:password [2018-01-25 21:30:08,605] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-3aef3b0/_view/test_view-3aef3b0?stale=ok [2018-01-25 21:30:08,737] - [task:2181] INFO - view : test_view-3aef3b0 was created successfully in ddoc: test_view-3aef3b0 [2018-01-25 21:30:09,762] - [rest_client:807] ERROR - GET http://cb-example-0003.cb-example.default.svc:8092//default/_design/test_view-e8b4520 body: headers: {'Content-Type': 'application/json', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 404 reason: not_found {"error":"not_found","reason":"missing"} auth: Administrator:password [2018-01-25 21:30:09,971] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-e8b4520/_view/test_view-e8b4520?stale=ok [2018-01-25 21:30:10,072] - [task:2181] INFO - view : test_view-e8b4520 was created successfully in ddoc: test_view-e8b4520 [2018-01-25 21:30:11,089] - [rest_client:807] ERROR - GET http://cb-example-0003.cb-example.default.svc:8092//default/_design/test_view-4205ca3 body: headers: {'Content-Type': 'application/json', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 404 reason: not_found {"error":"not_found","reason":"missing"} auth: Administrator:password [2018-01-25 21:30:11,146] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-4205ca3/_view/test_view-4205ca3?stale=ok [2018-01-25 21:30:11,252] - [task:2181] INFO - view : test_view-4205ca3 was created successfully in ddoc: test_view-4205ca3 [2018-01-25 21:30:11,335] - [viewquerytests:1980] INFO - create 2016 documents... [2018-01-25 21:30:11,375] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 default [2018-01-25 21:30:11,450] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 default [2018-01-25 21:30:11,506] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 default [2018-01-25 21:30:11,565] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:30:19,873] - [rebalance_helper:22] INFO - waiting for bucket default stat : ep_queue_size to match 0 on 172.17.0.8 [2018-01-25 21:30:20,010] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'cb-example-0003.cb-example.default.svc', 'password': 'password', 'port': 8091} [2018-01-25 21:30:20,011] - [data_helper:293] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:30:20,101] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'172.17.0.7', 'password': 'password', 'port': 8091} [2018-01-25 21:30:20,102] - [data_helper:293] INFO - creating direct client 172.17.0.7:11210 default [2018-01-25 21:30:20,176] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'172.17.0.5', 'password': 'password', 'port': 8091} [2018-01-25 21:30:20,176] - [data_helper:293] INFO - creating direct client 172.17.0.5:11210 default [2018-01-25 21:30:20,272] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'172.17.0.6', 'password': 'password', 'port': 8091} [2018-01-25 21:30:20,272] - [data_helper:293] INFO - creating direct client 172.17.0.6:11210 default [2018-01-25 21:30:20,371] - [rebalance_helper:46] INFO - 0 [2018-01-25 21:30:20,371] - [rebalance_helper:51] INFO - ep_queue_size : 0 [2018-01-25 21:30:20,371] - [rebalance_helper:22] INFO - waiting for bucket default stat : ep_flusher_todo to match 0 on 172.17.0.8 [2018-01-25 21:30:20,413] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'cb-example-0003.cb-example.default.svc', 'password': 'password', 'port': 8091} [2018-01-25 21:30:20,414] - [data_helper:293] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:30:20,494] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'172.17.0.7', 'password': 'password', 'port': 8091} [2018-01-25 21:30:20,494] - [data_helper:293] INFO - creating direct client 172.17.0.7:11210 default [2018-01-25 21:30:20,597] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'172.17.0.5', 'password': 'password', 'port': 8091} [2018-01-25 21:30:20,598] - [data_helper:293] INFO - creating direct client 172.17.0.5:11210 default [2018-01-25 21:30:20,663] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'172.17.0.6', 'password': 'password', 'port': 8091} [2018-01-25 21:30:20,663] - [data_helper:293] INFO - creating direct client 172.17.0.6:11210 default [2018-01-25 21:30:20,709] - [rebalance_helper:46] INFO - 0 [2018-01-25 21:30:20,709] - [rebalance_helper:51] INFO - ep_flusher_todo : 0 [2018-01-25 21:30:20,709] - [rebalance_helper:22] INFO - waiting for bucket default stat : ep_uncommitted_items to match 0 on 172.17.0.8 [2018-01-25 21:30:20,813] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'cb-example-0003.cb-example.default.svc', 'password': 'password', 'port': 8091} [2018-01-25 21:30:20,813] - [data_helper:293] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:30:20,886] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'172.17.0.7', 'password': 'password', 'port': 8091} [2018-01-25 21:30:20,886] - [data_helper:293] INFO - creating direct client 172.17.0.7:11210 default [2018-01-25 21:30:20,972] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'172.17.0.5', 'password': 'password', 'port': 8091} [2018-01-25 21:30:20,973] - [data_helper:293] INFO - creating direct client 172.17.0.5:11210 default [2018-01-25 21:30:21,046] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'172.17.0.6', 'password': 'password', 'port': 8091} [2018-01-25 21:30:21,046] - [data_helper:293] INFO - creating direct client 172.17.0.6:11210 default [2018-01-25 21:30:21,113] - [rebalance_helper:46] INFO - 0 [2018-01-25 21:30:21,114] - [rebalance_helper:51] INFO - ep_uncommitted_items : 0 [2018-01-25 21:30:21,115] - [viewquerytests:1990] INFO - LOAD IS FINISHED [2018-01-25 21:30:21,115] - [rebalance_helper:22] INFO - waiting for bucket default stat : ep_queue_size to match 0 on 172.17.0.8 [2018-01-25 21:30:21,150] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'cb-example-0003.cb-example.default.svc', 'password': 'password', 'port': 8091} [2018-01-25 21:30:21,150] - [data_helper:293] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:30:21,220] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'172.17.0.7', 'password': 'password', 'port': 8091} [2018-01-25 21:30:21,221] - [data_helper:293] INFO - creating direct client 172.17.0.7:11210 default [2018-01-25 21:30:21,303] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'172.17.0.5', 'password': 'password', 'port': 8091} [2018-01-25 21:30:21,303] - [data_helper:293] INFO - creating direct client 172.17.0.5:11210 default [2018-01-25 21:30:21,363] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'172.17.0.6', 'password': 'password', 'port': 8091} [2018-01-25 21:30:21,364] - [data_helper:293] INFO - creating direct client 172.17.0.6:11210 default [2018-01-25 21:30:21,412] - [rebalance_helper:46] INFO - 0 [2018-01-25 21:30:21,412] - [rebalance_helper:51] INFO - ep_queue_size : 0 [2018-01-25 21:30:21,412] - [rebalance_helper:22] INFO - waiting for bucket default stat : ep_flusher_todo to match 0 on 172.17.0.8 [2018-01-25 21:30:21,443] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'cb-example-0003.cb-example.default.svc', 'password': 'password', 'port': 8091} [2018-01-25 21:30:21,443] - [data_helper:293] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:30:21,507] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'172.17.0.7', 'password': 'password', 'port': 8091} [2018-01-25 21:30:21,507] - [data_helper:293] INFO - creating direct client 172.17.0.7:11210 default [2018-01-25 21:30:21,563] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'172.17.0.5', 'password': 'password', 'port': 8091} [2018-01-25 21:30:21,563] - [data_helper:293] INFO - creating direct client 172.17.0.5:11210 default [2018-01-25 21:30:21,626] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'172.17.0.6', 'password': 'password', 'port': 8091} [2018-01-25 21:30:21,626] - [data_helper:293] INFO - creating direct client 172.17.0.6:11210 default [2018-01-25 21:30:21,671] - [rebalance_helper:46] INFO - 0 [2018-01-25 21:30:21,671] - [rebalance_helper:51] INFO - ep_flusher_todo : 0 [2018-01-25 21:30:21,672] - [rebalance_helper:22] INFO - waiting for bucket default stat : ep_uncommitted_items to match 0 on 172.17.0.8 [2018-01-25 21:30:21,703] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'cb-example-0003.cb-example.default.svc', 'password': 'password', 'port': 8091} [2018-01-25 21:30:21,703] - [data_helper:293] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:30:21,767] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'172.17.0.7', 'password': 'password', 'port': 8091} [2018-01-25 21:30:21,767] - [data_helper:293] INFO - creating direct client 172.17.0.7:11210 default [2018-01-25 21:30:21,827] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'172.17.0.5', 'password': 'password', 'port': 8091} [2018-01-25 21:30:21,827] - [data_helper:293] INFO - creating direct client 172.17.0.5:11210 default [2018-01-25 21:30:21,902] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'172.17.0.6', 'password': 'password', 'port': 8091} [2018-01-25 21:30:21,903] - [data_helper:293] INFO - creating direct client 172.17.0.6:11210 default [2018-01-25 21:30:21,950] - [rebalance_helper:46] INFO - 0 [2018-01-25 21:30:21,950] - [rebalance_helper:51] INFO - ep_uncommitted_items : 0 [2018-01-25 21:30:21,951] - [rebalance_helper:22] INFO - waiting for bucket default stat : ep_queue_size to match 0 on 172.17.0.6 [2018-01-25 21:30:22,004] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'cb-example-0003.cb-example.default.svc', 'password': 'password', 'port': 8091} [2018-01-25 21:30:22,005] - [data_helper:293] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:30:22,081] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'172.17.0.7', 'password': 'password', 'port': 8091} [2018-01-25 21:30:22,082] - [data_helper:293] INFO - creating direct client 172.17.0.7:11210 default [2018-01-25 21:30:22,166] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'172.17.0.5', 'password': 'password', 'port': 8091} [2018-01-25 21:30:22,167] - [data_helper:293] INFO - creating direct client 172.17.0.5:11210 default [2018-01-25 21:30:22,239] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'172.17.0.6', 'password': 'password', 'port': 8091} [2018-01-25 21:30:22,239] - [data_helper:293] INFO - creating direct client 172.17.0.6:11210 default [2018-01-25 21:30:22,288] - [rebalance_helper:46] INFO - 0 [2018-01-25 21:30:22,288] - [rebalance_helper:51] INFO - ep_queue_size : 0 [2018-01-25 21:30:22,289] - [rebalance_helper:22] INFO - waiting for bucket default stat : ep_flusher_todo to match 0 on 172.17.0.6 [2018-01-25 21:30:22,328] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'cb-example-0003.cb-example.default.svc', 'password': 'password', 'port': 8091} [2018-01-25 21:30:22,328] - [data_helper:293] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:30:22,404] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'172.17.0.7', 'password': 'password', 'port': 8091} [2018-01-25 21:30:22,404] - [data_helper:293] INFO - creating direct client 172.17.0.7:11210 default [2018-01-25 21:30:22,480] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'172.17.0.5', 'password': 'password', 'port': 8091} [2018-01-25 21:30:22,480] - [data_helper:293] INFO - creating direct client 172.17.0.5:11210 default [2018-01-25 21:30:22,541] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'172.17.0.6', 'password': 'password', 'port': 8091} [2018-01-25 21:30:22,541] - [data_helper:293] INFO - creating direct client 172.17.0.6:11210 default [2018-01-25 21:30:22,603] - [rebalance_helper:46] INFO - 0 [2018-01-25 21:30:22,603] - [rebalance_helper:51] INFO - ep_flusher_todo : 0 [2018-01-25 21:30:22,603] - [rebalance_helper:22] INFO - waiting for bucket default stat : ep_uncommitted_items to match 0 on 172.17.0.6 [2018-01-25 21:30:22,643] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'cb-example-0003.cb-example.default.svc', 'password': 'password', 'port': 8091} [2018-01-25 21:30:22,643] - [data_helper:293] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:30:22,714] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'172.17.0.7', 'password': 'password', 'port': 8091} [2018-01-25 21:30:22,714] - [data_helper:293] INFO - creating direct client 172.17.0.7:11210 default [2018-01-25 21:30:22,774] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'172.17.0.5', 'password': 'password', 'port': 8091} [2018-01-25 21:30:22,774] - [data_helper:293] INFO - creating direct client 172.17.0.5:11210 default [2018-01-25 21:30:22,831] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'172.17.0.6', 'password': 'password', 'port': 8091} [2018-01-25 21:30:22,832] - [data_helper:293] INFO - creating direct client 172.17.0.6:11210 default [2018-01-25 21:30:22,871] - [rebalance_helper:46] INFO - 0 [2018-01-25 21:30:22,872] - [rebalance_helper:51] INFO - ep_uncommitted_items : 0 [2018-01-25 21:30:22,872] - [rebalance_helper:22] INFO - waiting for bucket default stat : ep_queue_size to match 0 on 172.17.0.7 [2018-01-25 21:30:22,907] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'cb-example-0003.cb-example.default.svc', 'password': 'password', 'port': 8091} [2018-01-25 21:30:22,907] - [data_helper:293] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:30:22,987] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'172.17.0.7', 'password': 'password', 'port': 8091} [2018-01-25 21:30:22,987] - [data_helper:293] INFO - creating direct client 172.17.0.7:11210 default [2018-01-25 21:30:23,056] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'172.17.0.5', 'password': 'password', 'port': 8091} [2018-01-25 21:30:23,056] - [data_helper:293] INFO - creating direct client 172.17.0.5:11210 default [2018-01-25 21:30:23,128] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'172.17.0.6', 'password': 'password', 'port': 8091} [2018-01-25 21:30:23,128] - [data_helper:293] INFO - creating direct client 172.17.0.6:11210 default [2018-01-25 21:30:23,179] - [rebalance_helper:46] INFO - 0 [2018-01-25 21:30:23,179] - [rebalance_helper:51] INFO - ep_queue_size : 0 [2018-01-25 21:30:23,180] - [rebalance_helper:22] INFO - waiting for bucket default stat : ep_flusher_todo to match 0 on 172.17.0.7 [2018-01-25 21:30:23,215] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'cb-example-0003.cb-example.default.svc', 'password': 'password', 'port': 8091} [2018-01-25 21:30:23,216] - [data_helper:293] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:30:23,305] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'172.17.0.7', 'password': 'password', 'port': 8091} [2018-01-25 21:30:23,306] - [data_helper:293] INFO - creating direct client 172.17.0.7:11210 default [2018-01-25 21:30:23,365] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'172.17.0.5', 'password': 'password', 'port': 8091} [2018-01-25 21:30:23,365] - [data_helper:293] INFO - creating direct client 172.17.0.5:11210 default [2018-01-25 21:30:23,426] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'172.17.0.6', 'password': 'password', 'port': 8091} [2018-01-25 21:30:23,494] - [data_helper:293] INFO - creating direct client 172.17.0.6:11210 default [2018-01-25 21:30:23,539] - [rebalance_helper:46] INFO - 0 [2018-01-25 21:30:23,539] - [rebalance_helper:51] INFO - ep_flusher_todo : 0 [2018-01-25 21:30:23,539] - [rebalance_helper:22] INFO - waiting for bucket default stat : ep_uncommitted_items to match 0 on 172.17.0.7 [2018-01-25 21:30:23,570] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'cb-example-0003.cb-example.default.svc', 'password': 'password', 'port': 8091} [2018-01-25 21:30:23,570] - [data_helper:293] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:30:23,658] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'172.17.0.7', 'password': 'password', 'port': 8091} [2018-01-25 21:30:23,658] - [data_helper:293] INFO - creating direct client 172.17.0.7:11210 default [2018-01-25 21:30:23,717] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'172.17.0.5', 'password': 'password', 'port': 8091} [2018-01-25 21:30:23,718] - [data_helper:293] INFO - creating direct client 172.17.0.5:11210 default [2018-01-25 21:30:23,778] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'172.17.0.6', 'password': 'password', 'port': 8091} [2018-01-25 21:30:23,778] - [data_helper:293] INFO - creating direct client 172.17.0.6:11210 default [2018-01-25 21:30:23,829] - [rebalance_helper:46] INFO - 0 [2018-01-25 21:30:23,830] - [rebalance_helper:51] INFO - ep_uncommitted_items : 0 [2018-01-25 21:30:23,830] - [rebalance_helper:22] INFO - waiting for bucket default stat : ep_queue_size to match 0 on 172.17.0.5 [2018-01-25 21:30:23,865] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'cb-example-0003.cb-example.default.svc', 'password': 'password', 'port': 8091} [2018-01-25 21:30:23,865] - [data_helper:293] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:30:24,062] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'172.17.0.7', 'password': 'password', 'port': 8091} [2018-01-25 21:30:24,062] - [data_helper:293] INFO - creating direct client 172.17.0.7:11210 default [2018-01-25 21:30:24,156] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'172.17.0.5', 'password': 'password', 'port': 8091} [2018-01-25 21:30:24,156] - [data_helper:293] INFO - creating direct client 172.17.0.5:11210 default [2018-01-25 21:30:24,240] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'172.17.0.6', 'password': 'password', 'port': 8091} [2018-01-25 21:30:24,240] - [data_helper:293] INFO - creating direct client 172.17.0.6:11210 default [2018-01-25 21:30:24,310] - [rebalance_helper:46] INFO - 0 [2018-01-25 21:30:24,310] - [rebalance_helper:51] INFO - ep_queue_size : 0 [2018-01-25 21:30:24,311] - [rebalance_helper:22] INFO - waiting for bucket default stat : ep_flusher_todo to match 0 on 172.17.0.5 [2018-01-25 21:30:24,354] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'cb-example-0003.cb-example.default.svc', 'password': 'password', 'port': 8091} [2018-01-25 21:30:24,354] - [data_helper:293] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:30:24,438] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'172.17.0.7', 'password': 'password', 'port': 8091} [2018-01-25 21:30:24,438] - [data_helper:293] INFO - creating direct client 172.17.0.7:11210 default [2018-01-25 21:30:24,503] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'172.17.0.5', 'password': 'password', 'port': 8091} [2018-01-25 21:30:24,503] - [data_helper:293] INFO - creating direct client 172.17.0.5:11210 default [2018-01-25 21:30:24,572] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'172.17.0.6', 'password': 'password', 'port': 8091} [2018-01-25 21:30:24,579] - [data_helper:293] INFO - creating direct client 172.17.0.6:11210 default [2018-01-25 21:30:24,644] - [rebalance_helper:46] INFO - 0 [2018-01-25 21:30:24,644] - [rebalance_helper:51] INFO - ep_flusher_todo : 0 [2018-01-25 21:30:24,645] - [rebalance_helper:22] INFO - waiting for bucket default stat : ep_uncommitted_items to match 0 on 172.17.0.5 [2018-01-25 21:30:24,690] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'cb-example-0003.cb-example.default.svc', 'password': 'password', 'port': 8091} [2018-01-25 21:30:24,690] - [data_helper:293] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:30:24,792] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'172.17.0.7', 'password': 'password', 'port': 8091} [2018-01-25 21:30:24,793] - [data_helper:293] INFO - creating direct client 172.17.0.7:11210 default [2018-01-25 21:30:24,890] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'172.17.0.5', 'password': 'password', 'port': 8091} [2018-01-25 21:30:24,891] - [data_helper:293] INFO - creating direct client 172.17.0.5:11210 default [2018-01-25 21:30:24,988] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': u'172.17.0.6', 'password': 'password', 'port': 8091} [2018-01-25 21:30:24,988] - [data_helper:293] INFO - creating direct client 172.17.0.6:11210 default [2018-01-25 21:30:25,061] - [rebalance_helper:46] INFO - 0 [2018-01-25 21:30:25,062] - [rebalance_helper:51] INFO - ep_uncommitted_items : 0 [2018-01-25 21:30:25,062] - [viewquerytests:2143] INFO - DDoc=test_view-4e17ebd; View={"test_view-4e17ebd": {"map": "function (doc) { if(doc.job_title !== undefined) emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] ); }"}};: query={'stale': 'false'}: Generate results [2018-01-25 21:30:25,064] - [viewquerytests:2143] INFO - DDoc=test_view-52950e3; View={"test_view-52950e3": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^UI \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};: query={'stale': 'false'}: Generate results [2018-01-25 21:30:25,065] - [viewquerytests:2143] INFO - DDoc=test_view-edcbe72; View={"test_view-edcbe72": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^System \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};: query={'stale': 'false'}: Generate results [2018-01-25 21:30:25,066] - [viewquerytests:2143] INFO - DDoc=test_view-3aef3b0; View={"test_view-3aef3b0": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^Senior \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};: query={'stale': 'false'}: Generate results [2018-01-25 21:30:25,066] - [viewquerytests:2143] INFO - DDoc=test_view-e8b4520; View={"test_view-e8b4520": {"map": "function (doc) { if(doc.job_title !== undefined) emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] ); }", "reduce": "_count"}};: query={'stale': 'false'}: Generate results [2018-01-25 21:30:25,067] - [viewquerytests:2047] INFO - Current amount of threads 6 [2018-01-25 21:30:25,067] - [viewquerytests:2143] INFO - DDoc=test_view-4205ca3; View={"test_view-4205ca3": {"map": "function (doc, meta) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^System \"); if(doc.job_title.match(myregexp)) { emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};: query={'stale': 'false'}: Generate results [2018-01-25 21:30:25,505] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:25,684] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:25,866] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:26,073] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:26,248] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:26,428] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:26,429] - [viewquerytests:2159] INFO - DDoc=test_view-edcbe72; View={"test_view-edcbe72": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^System \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};:{'stale': 'false'}: Generated results contains 672 items [2018-01-25 21:30:26,431] - [viewquerytests:2159] INFO - DDoc=test_view-52950e3; View={"test_view-52950e3": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^UI \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};:{'stale': 'false'}: Generated results contains 672 items [2018-01-25 21:30:26,432] - [viewquerytests:2159] INFO - DDoc=test_view-e8b4520; View={"test_view-e8b4520": {"map": "function (doc) { if(doc.job_title !== undefined) emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] ); }", "reduce": "_count"}};:{'stale': 'false'}: Generated results contains 1 items [2018-01-25 21:30:26,430] - [viewquerytests:2159] INFO - DDoc=test_view-4e17ebd; View={"test_view-4e17ebd": {"map": "function (doc) { if(doc.job_title !== undefined) emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] ); }"}};:{'stale': 'false'}: Generated results contains 2016 items [2018-01-25 21:30:26,435] - [viewquerytests:2159] INFO - DDoc=test_view-3aef3b0; View={"test_view-3aef3b0": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^Senior \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};:{'stale': 'false'}: Generated results contains 672 items [2018-01-25 21:30:26,443] - [viewquerytests:2159] INFO - DDoc=test_view-4205ca3; View={"test_view-4205ca3": {"map": "function (doc, meta) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^System \"); if(doc.job_title.match(myregexp)) { emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};:{'stale': 'false'}: Generated results contains 672 items [2018-01-25 21:30:27,433] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-e8b4520/_view/test_view-e8b4520?stale=false&connection_timeout=60000 [2018-01-25 21:30:28,220] - [task:2724] INFO - view test_view-e8b4520, query {'stale': 'false', 'connection_timeout': 60000}: expected- 1, actual -1 [2018-01-25 21:30:28,220] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-4e17ebd/_view/test_view-4e17ebd?stale=false&connection_timeout=60000 [2018-01-25 21:30:29,149] - [task:2724] INFO - view test_view-4e17ebd, query {'stale': 'false', 'connection_timeout': 60000}: expected- 2016, actual -2016 [2018-01-25 21:30:29,150] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-52950e3/_view/test_view-52950e3?stale=false&connection_timeout=60000 [2018-01-25 21:30:29,938] - [task:2724] INFO - view test_view-52950e3, query {'stale': 'false', 'connection_timeout': 60000}: expected- 672, actual -672 [2018-01-25 21:30:29,939] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-edcbe72/_view/test_view-edcbe72?stale=false&connection_timeout=60000 [2018-01-25 21:30:30,602] - [task:2724] INFO - view test_view-edcbe72, query {'stale': 'false', 'connection_timeout': 60000}: expected- 672, actual -672 [2018-01-25 21:30:30,603] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-3aef3b0/_view/test_view-3aef3b0?stale=false&connection_timeout=60000 [2018-01-25 21:30:31,201] - [task:2724] INFO - view test_view-3aef3b0, query {'stale': 'false', 'connection_timeout': 60000}: expected- 672, actual -672 [2018-01-25 21:30:31,202] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-4205ca3/_view/test_view-4205ca3?stale=false&connection_timeout=60000 [2018-01-25 21:30:32,445] - [task:2724] INFO - view test_view-4205ca3, query {'stale': 'false', 'connection_timeout': 60000}: expected- 672, actual -672 [2018-01-25 21:30:32,445] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'stale': 'false', 'connection_timeout': 60000}, server=172.17.0.8 Key None: expected value 2016, actual: 2016 [2018-01-25 21:30:32,445] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'stale': 'false', 'connection_timeout': 60000}, server=172.17.0.8 Key None: expected value 2016, actual: 2016 [2018-01-25 21:30:32,446] - [viewquerytests:2143] INFO - DDoc=test_view-e8b4520; View={"test_view-e8b4520": {"map": "function (doc) { if(doc.job_title !== undefined) emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] ); }", "reduce": "_count"}};: query={'stale': 'ok'}: Generate results [2018-01-25 21:30:32,446] - [viewquerytests:2143] INFO - DDoc=test_view-edcbe72; View={"test_view-edcbe72": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^System \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};: query={'stale': 'ok'}: Generate results [2018-01-25 21:30:32,447] - [viewquerytests:2143] INFO - DDoc=test_view-52950e3; View={"test_view-52950e3": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^UI \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};: query={'stale': 'ok'}: Generate results [2018-01-25 21:30:32,447] - [viewquerytests:2143] INFO - DDoc=test_view-4e17ebd; View={"test_view-4e17ebd": {"map": "function (doc) { if(doc.job_title !== undefined) emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] ); }"}};: query={'stale': 'ok'}: Generate results [2018-01-25 21:30:32,448] - [viewquerytests:2143] INFO - DDoc=test_view-3aef3b0; View={"test_view-3aef3b0": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^Senior \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};: query={'stale': 'ok'}: Generate results [2018-01-25 21:30:32,448] - [viewquerytests:2143] INFO - DDoc=test_view-4205ca3; View={"test_view-4205ca3": {"map": "function (doc, meta) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^System \"); if(doc.job_title.match(myregexp)) { emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};: query={'stale': 'ok'}: Generate results [2018-01-25 21:30:32,899] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:33,158] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:33,412] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:33,412] - [viewquerytests:2159] INFO - DDoc=test_view-edcbe72; View={"test_view-edcbe72": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^System \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};:{'stale': 'ok'}: Generated results contains 672 items [2018-01-25 21:30:33,413] - [viewquerytests:2159] INFO - DDoc=test_view-52950e3; View={"test_view-52950e3": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^UI \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};:{'stale': 'ok'}: Generated results contains 672 items [2018-01-25 21:30:33,417] - [viewquerytests:2159] INFO - DDoc=test_view-e8b4520; View={"test_view-e8b4520": {"map": "function (doc) { if(doc.job_title !== undefined) emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] ); }", "reduce": "_count"}};:{'stale': 'ok'}: Generated results contains 1 items [2018-01-25 21:30:33,629] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:33,991] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:34,282] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:34,282] - [viewquerytests:2159] INFO - DDoc=test_view-4e17ebd; View={"test_view-4e17ebd": {"map": "function (doc) { if(doc.job_title !== undefined) emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] ); }"}};:{'stale': 'ok'}: Generated results contains 2016 items [2018-01-25 21:30:34,283] - [viewquerytests:2159] INFO - DDoc=test_view-3aef3b0; View={"test_view-3aef3b0": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^Senior \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};:{'stale': 'ok'}: Generated results contains 672 items [2018-01-25 21:30:34,283] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-52950e3/_view/test_view-52950e3?stale=ok&connection_timeout=60000 [2018-01-25 21:30:34,284] - [viewquerytests:2159] INFO - DDoc=test_view-4205ca3; View={"test_view-4205ca3": {"map": "function (doc, meta) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^System \"); if(doc.job_title.match(myregexp)) { emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};:{'stale': 'ok'}: Generated results contains 672 items [2018-01-25 21:30:34,336] - [task:2724] INFO - view test_view-52950e3, query {'stale': 'ok', 'connection_timeout': 60000}: expected- 672, actual -672 [2018-01-25 21:30:34,337] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-edcbe72/_view/test_view-edcbe72?stale=ok&connection_timeout=60000 [2018-01-25 21:30:34,375] - [task:2724] INFO - view test_view-edcbe72, query {'stale': 'ok', 'connection_timeout': 60000}: expected- 672, actual -672 [2018-01-25 21:30:34,375] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-e8b4520/_view/test_view-e8b4520?stale=ok&connection_timeout=60000 [2018-01-25 21:30:34,395] - [task:2724] INFO - view test_view-e8b4520, query {'stale': 'ok', 'connection_timeout': 60000}: expected- 1, actual -1 [2018-01-25 21:30:34,395] - [viewquerytests:2143] INFO - DDoc=test_view-52950e3; View={"test_view-52950e3": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^UI \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};: query={'stale': 'update_after'}: Generate results [2018-01-25 21:30:34,396] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'stale': 'ok', 'connection_timeout': 60000}, server=172.17.0.8 Key None: expected value 2016, actual: 2016 [2018-01-25 21:30:34,396] - [viewquerytests:2143] INFO - DDoc=test_view-edcbe72; View={"test_view-edcbe72": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^System \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};: query={'stale': 'update_after'}: Generate results [2018-01-25 21:30:34,396] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'stale': 'ok', 'connection_timeout': 60000}, server=172.17.0.8 Key None: expected value 2016, actual: 2016 [2018-01-25 21:30:34,397] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-4e17ebd/_view/test_view-4e17ebd?stale=ok&connection_timeout=60000 [2018-01-25 21:30:34,397] - [viewquerytests:2143] INFO - DDoc=test_view-e8b4520; View={"test_view-e8b4520": {"map": "function (doc) { if(doc.job_title !== undefined) emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] ); }", "reduce": "_count"}};: query={'stale': 'update_after'}: Generate results [2018-01-25 21:30:34,564] - [task:2724] INFO - view test_view-4e17ebd, query {'stale': 'ok', 'connection_timeout': 60000}: expected- 2016, actual -2016 [2018-01-25 21:30:34,565] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-4205ca3/_view/test_view-4205ca3?stale=ok&connection_timeout=60000 [2018-01-25 21:30:35,093] - [task:2724] INFO - view test_view-4205ca3, query {'stale': 'ok', 'connection_timeout': 60000}: expected- 672, actual -672 [2018-01-25 21:30:35,093] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-3aef3b0/_view/test_view-3aef3b0?stale=ok&connection_timeout=60000 [2018-01-25 21:30:35,181] - [task:2724] INFO - view test_view-3aef3b0, query {'stale': 'ok', 'connection_timeout': 60000}: expected- 672, actual -672 [2018-01-25 21:30:35,183] - [viewquerytests:2143] INFO - DDoc=test_view-4e17ebd; View={"test_view-4e17ebd": {"map": "function (doc) { if(doc.job_title !== undefined) emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] ); }"}};: query={'stale': 'update_after'}: Generate results [2018-01-25 21:30:35,183] - [viewquerytests:2143] INFO - DDoc=test_view-4205ca3; View={"test_view-4205ca3": {"map": "function (doc, meta) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^System \"); if(doc.job_title.match(myregexp)) { emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};: query={'stale': 'update_after'}: Generate results [2018-01-25 21:30:35,184] - [viewquerytests:2143] INFO - DDoc=test_view-3aef3b0; View={"test_view-3aef3b0": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^Senior \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};: query={'stale': 'update_after'}: Generate results [2018-01-25 21:30:35,610] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:35,865] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:36,128] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:36,130] - [viewquerytests:2159] INFO - DDoc=test_view-e8b4520; View={"test_view-e8b4520": {"map": "function (doc) { if(doc.job_title !== undefined) emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] ); }", "reduce": "_count"}};:{'stale': 'update_after'}: Generated results contains 1 items [2018-01-25 21:30:36,130] - [viewquerytests:2159] INFO - DDoc=test_view-edcbe72; View={"test_view-edcbe72": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^System \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};:{'stale': 'update_after'}: Generated results contains 672 items [2018-01-25 21:30:36,132] - [viewquerytests:2159] INFO - DDoc=test_view-52950e3; View={"test_view-52950e3": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^UI \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};:{'stale': 'update_after'}: Generated results contains 672 items [2018-01-25 21:30:36,429] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:36,794] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:37,516] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:37,520] - [viewquerytests:2159] INFO - DDoc=test_view-3aef3b0; View={"test_view-3aef3b0": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^Senior \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};:{'stale': 'update_after'}: Generated results contains 672 items [2018-01-25 21:30:37,520] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-e8b4520/_view/test_view-e8b4520?stale=update_after&connection_timeout=60000 [2018-01-25 21:30:37,520] - [viewquerytests:2159] INFO - DDoc=test_view-4e17ebd; View={"test_view-4e17ebd": {"map": "function (doc) { if(doc.job_title !== undefined) emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] ); }"}};:{'stale': 'update_after'}: Generated results contains 2016 items [2018-01-25 21:30:37,519] - [viewquerytests:2159] INFO - DDoc=test_view-4205ca3; View={"test_view-4205ca3": {"map": "function (doc, meta) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^System \"); if(doc.job_title.match(myregexp)) { emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};:{'stale': 'update_after'}: Generated results contains 672 items [2018-01-25 21:30:37,561] - [task:2724] INFO - view test_view-e8b4520, query {'stale': 'update_after', 'connection_timeout': 60000}: expected- 1, actual -1 [2018-01-25 21:30:37,563] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-edcbe72/_view/test_view-edcbe72?stale=update_after&connection_timeout=60000 [2018-01-25 21:30:37,661] - [task:2724] INFO - view test_view-edcbe72, query {'stale': 'update_after', 'connection_timeout': 60000}: expected- 672, actual -672 [2018-01-25 21:30:37,661] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-52950e3/_view/test_view-52950e3?stale=update_after&connection_timeout=60000 [2018-01-25 21:30:37,697] - [task:2724] INFO - view test_view-52950e3, query {'stale': 'update_after', 'connection_timeout': 60000}: expected- 672, actual -672 [2018-01-25 21:30:37,698] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'stale': 'update_after', 'connection_timeout': 60000}, server=172.17.0.8 Key None: expected value 2016, actual: 2016 [2018-01-25 21:30:37,698] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'stale': 'update_after', 'connection_timeout': 60000}, server=172.17.0.8 Key None: expected value 2016, actual: 2016 [2018-01-25 21:30:37,699] - [viewquerytests:2143] INFO - DDoc=test_view-e8b4520; View={"test_view-e8b4520": {"map": "function (doc) { if(doc.job_title !== undefined) emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] ); }", "reduce": "_count"}};: query={'startkey': '[2008,7,null]'}: Generate results [2018-01-25 21:30:37,698] - [viewquerytests:2143] INFO - DDoc=test_view-edcbe72; View={"test_view-edcbe72": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^System \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};: query={'startkey': '[2008,7,null]'}: Generate results [2018-01-25 21:30:37,699] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-3aef3b0/_view/test_view-3aef3b0?stale=update_after&connection_timeout=60000 [2018-01-25 21:30:37,699] - [viewquerytests:2143] INFO - DDoc=test_view-52950e3; View={"test_view-52950e3": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^UI \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};: query={'startkey': '[2008,7,null]'}: Generate results [2018-01-25 21:30:37,738] - [task:2724] INFO - view test_view-3aef3b0, query {'stale': 'update_after', 'connection_timeout': 60000}: expected- 672, actual -672 [2018-01-25 21:30:37,739] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-4e17ebd/_view/test_view-4e17ebd?stale=update_after&connection_timeout=60000 [2018-01-25 21:30:37,838] - [task:2724] INFO - view test_view-4e17ebd, query {'stale': 'update_after', 'connection_timeout': 60000}: expected- 2016, actual -2016 [2018-01-25 21:30:37,839] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-4205ca3/_view/test_view-4205ca3?stale=update_after&connection_timeout=60000 [2018-01-25 21:30:37,888] - [task:2724] INFO - view test_view-4205ca3, query {'stale': 'update_after', 'connection_timeout': 60000}: expected- 672, actual -672 [2018-01-25 21:30:37,892] - [viewquerytests:2143] INFO - DDoc=test_view-4e17ebd; View={"test_view-4e17ebd": {"map": "function (doc) { if(doc.job_title !== undefined) emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] ); }"}};: query={'startkey': '[2008,7,null]'}: Generate results [2018-01-25 21:30:37,893] - [viewquerytests:2143] INFO - DDoc=test_view-4205ca3; View={"test_view-4205ca3": {"map": "function (doc, meta) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^System \"); if(doc.job_title.match(myregexp)) { emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};: query={'startkey': '[2008,7,null]'}: Generate results [2018-01-25 21:30:37,894] - [viewquerytests:2143] INFO - DDoc=test_view-3aef3b0; View={"test_view-3aef3b0": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^Senior \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};: query={'startkey': '[2008,7,null]'}: Generate results [2018-01-25 21:30:38,110] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:38,335] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:38,588] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:38,589] - [viewquerytests:2159] INFO - DDoc=test_view-edcbe72; View={"test_view-edcbe72": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^System \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};:{'startkey': '[2008,7,null]'}: Generated results contains 336 items [2018-01-25 21:30:38,590] - [viewquerytests:2159] INFO - DDoc=test_view-e8b4520; View={"test_view-e8b4520": {"map": "function (doc) { if(doc.job_title !== undefined) emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] ); }", "reduce": "_count"}};:{'startkey': '[2008,7,null]'}: Generated results contains 1 items [2018-01-25 21:30:38,589] - [viewquerytests:2159] INFO - DDoc=test_view-52950e3; View={"test_view-52950e3": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^UI \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};:{'startkey': '[2008,7,null]'}: Generated results contains 336 items [2018-01-25 21:30:38,881] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:39,146] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:39,349] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:39,349] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-52950e3/_view/test_view-52950e3?startkey=%5B2008%2C7%2Cnull%5D&connection_timeout=60000 [2018-01-25 21:30:39,349] - [viewquerytests:2159] INFO - DDoc=test_view-4e17ebd; View={"test_view-4e17ebd": {"map": "function (doc) { if(doc.job_title !== undefined) emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] ); }"}};:{'startkey': '[2008,7,null]'}: Generated results contains 1008 items [2018-01-25 21:30:39,350] - [viewquerytests:2159] INFO - DDoc=test_view-4205ca3; View={"test_view-4205ca3": {"map": "function (doc, meta) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^System \"); if(doc.job_title.match(myregexp)) { emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};:{'startkey': '[2008,7,null]'}: Generated results contains 336 items [2018-01-25 21:30:39,351] - [viewquerytests:2159] INFO - DDoc=test_view-3aef3b0; View={"test_view-3aef3b0": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^Senior \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};:{'startkey': '[2008,7,null]'}: Generated results contains 336 items [2018-01-25 21:30:39,391] - [task:2724] INFO - view test_view-52950e3, query {'startkey': '[2008,7,null]', 'connection_timeout': 60000}: expected- 336, actual -336 [2018-01-25 21:30:39,391] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-e8b4520/_view/test_view-e8b4520?startkey=%5B2008%2C7%2Cnull%5D&connection_timeout=60000 [2018-01-25 21:30:39,402] - [task:2724] INFO - view test_view-e8b4520, query {'startkey': '[2008,7,null]', 'connection_timeout': 60000}: expected- 1, actual -1 [2018-01-25 21:30:39,402] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-edcbe72/_view/test_view-edcbe72?startkey=%5B2008%2C7%2Cnull%5D&connection_timeout=60000 [2018-01-25 21:30:39,425] - [task:2724] INFO - view test_view-edcbe72, query {'startkey': '[2008,7,null]', 'connection_timeout': 60000}: expected- 336, actual -336 [2018-01-25 21:30:39,426] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'startkey': '[2008,7,null]', 'connection_timeout': 60000}, server=172.17.0.8 Key None: expected value 1008, actual: 1008 [2018-01-25 21:30:39,426] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'startkey': '[2008,7,null]', 'connection_timeout': 60000}, server=172.17.0.8 Key None: expected value 1008, actual: 1008 [2018-01-25 21:30:39,426] - [viewquerytests:2143] INFO - DDoc=test_view-52950e3; View={"test_view-52950e3": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^UI \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};: query={'startkey': '[2008,0,1]', 'endkey': '[2008,7,1]', 'inclusive_end': 'false'}: Generate results [2018-01-25 21:30:39,427] - [viewquerytests:2143] INFO - DDoc=test_view-e8b4520; View={"test_view-e8b4520": {"map": "function (doc) { if(doc.job_title !== undefined) emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] ); }", "reduce": "_count"}};: query={'startkey': '[2008,0,1]', 'endkey': '[2008,7,1]', 'inclusive_end': 'false'}: Generate results [2018-01-25 21:30:39,427] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-4205ca3/_view/test_view-4205ca3?startkey=%5B2008%2C7%2Cnull%5D&connection_timeout=60000 [2018-01-25 21:30:39,427] - [viewquerytests:2143] INFO - DDoc=test_view-edcbe72; View={"test_view-edcbe72": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^System \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};: query={'startkey': '[2008,0,1]', 'endkey': '[2008,7,1]', 'inclusive_end': 'false'}: Generate results [2018-01-25 21:30:39,448] - [task:2724] INFO - view test_view-4205ca3, query {'startkey': '[2008,7,null]', 'connection_timeout': 60000}: expected- 336, actual -336 [2018-01-25 21:30:39,449] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-3aef3b0/_view/test_view-3aef3b0?startkey=%5B2008%2C7%2Cnull%5D&connection_timeout=60000 [2018-01-25 21:30:39,470] - [task:2724] INFO - view test_view-3aef3b0, query {'startkey': '[2008,7,null]', 'connection_timeout': 60000}: expected- 336, actual -336 [2018-01-25 21:30:39,470] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-4e17ebd/_view/test_view-4e17ebd?startkey=%5B2008%2C7%2Cnull%5D&connection_timeout=60000 [2018-01-25 21:30:39,508] - [task:2724] INFO - view test_view-4e17ebd, query {'startkey': '[2008,7,null]', 'connection_timeout': 60000}: expected- 1008, actual -1008 [2018-01-25 21:30:39,508] - [viewquerytests:2143] INFO - DDoc=test_view-4205ca3; View={"test_view-4205ca3": {"map": "function (doc, meta) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^System \"); if(doc.job_title.match(myregexp)) { emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};: query={'startkey': '[2008,0,1]', 'endkey': '[2008,7,1]', 'inclusive_end': 'false'}: Generate results [2018-01-25 21:30:39,509] - [viewquerytests:2143] INFO - DDoc=test_view-3aef3b0; View={"test_view-3aef3b0": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^Senior \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};: query={'startkey': '[2008,0,1]', 'endkey': '[2008,7,1]', 'inclusive_end': 'false'}: Generate results [2018-01-25 21:30:39,511] - [viewquerytests:2143] INFO - DDoc=test_view-4e17ebd; View={"test_view-4e17ebd": {"map": "function (doc) { if(doc.job_title !== undefined) emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] ); }"}};: query={'startkey': '[2008,0,1]', 'endkey': '[2008,7,1]', 'inclusive_end': 'false'}: Generate results [2018-01-25 21:30:39,713] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:39,881] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:40,030] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:40,031] - [viewquerytests:2159] INFO - DDoc=test_view-52950e3; View={"test_view-52950e3": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^UI \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};:{'startkey': '[2008,0,1]', 'endkey': '[2008,7,1]', 'inclusive_end': 'false'}: Generated results contains 336 items [2018-01-25 21:30:40,032] - [viewquerytests:2159] INFO - DDoc=test_view-e8b4520; View={"test_view-e8b4520": {"map": "function (doc) { if(doc.job_title !== undefined) emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] ); }", "reduce": "_count"}};:{'startkey': '[2008,0,1]', 'endkey': '[2008,7,1]', 'inclusive_end': 'false'}: Generated results contains 1 items [2018-01-25 21:30:40,033] - [viewquerytests:2159] INFO - DDoc=test_view-edcbe72; View={"test_view-edcbe72": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^System \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};:{'startkey': '[2008,0,1]', 'endkey': '[2008,7,1]', 'inclusive_end': 'false'}: Generated results contains 336 items [2018-01-25 21:30:40,250] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:40,441] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:40,631] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:40,632] - [viewquerytests:2159] INFO - DDoc=test_view-4205ca3; View={"test_view-4205ca3": {"map": "function (doc, meta) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^System \"); if(doc.job_title.match(myregexp)) { emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};:{'startkey': '[2008,0,1]', 'endkey': '[2008,7,1]', 'inclusive_end': 'false'}: Generated results contains 336 items [2018-01-25 21:30:40,634] - [viewquerytests:2159] INFO - DDoc=test_view-4e17ebd; View={"test_view-4e17ebd": {"map": "function (doc) { if(doc.job_title !== undefined) emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] ); }"}};:{'startkey': '[2008,0,1]', 'endkey': '[2008,7,1]', 'inclusive_end': 'false'}: Generated results contains 1008 items [2018-01-25 21:30:40,634] - [viewquerytests:2159] INFO - DDoc=test_view-3aef3b0; View={"test_view-3aef3b0": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^Senior \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};:{'startkey': '[2008,0,1]', 'endkey': '[2008,7,1]', 'inclusive_end': 'false'}: Generated results contains 336 items [2018-01-25 21:30:40,637] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-e8b4520/_view/test_view-e8b4520?startkey=%5B2008%2C0%2C1%5D&endkey=%5B2008%2C7%2C1%5D&connection_timeout=60000&inclusive_end=false [2018-01-25 21:30:40,659] - [task:2724] INFO - view test_view-e8b4520, query {'startkey': '[2008,0,1]', 'endkey': '[2008,7,1]', 'connection_timeout': 60000, 'inclusive_end': 'false'}: expected- 1, actual -1 [2018-01-25 21:30:40,659] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-52950e3/_view/test_view-52950e3?startkey=%5B2008%2C0%2C1%5D&endkey=%5B2008%2C7%2C1%5D&connection_timeout=60000&inclusive_end=false [2018-01-25 21:30:40,696] - [task:2724] INFO - view test_view-52950e3, query {'startkey': '[2008,0,1]', 'endkey': '[2008,7,1]', 'connection_timeout': 60000, 'inclusive_end': 'false'}: expected- 336, actual -336 [2018-01-25 21:30:40,696] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-edcbe72/_view/test_view-edcbe72?startkey=%5B2008%2C0%2C1%5D&endkey=%5B2008%2C7%2C1%5D&connection_timeout=60000&inclusive_end=false [2018-01-25 21:30:40,718] - [task:2724] INFO - view test_view-edcbe72, query {'startkey': '[2008,0,1]', 'endkey': '[2008,7,1]', 'connection_timeout': 60000, 'inclusive_end': 'false'}: expected- 336, actual -336 [2018-01-25 21:30:40,719] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'startkey': '[2008,0,1]', 'endkey': '[2008,7,1]', 'connection_timeout': 60000, 'inclusive_end': 'false'}, server=172.17.0.8 Key None: expected value 1008, actual: 1008 [2018-01-25 21:30:40,719] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'startkey': '[2008,0,1]', 'endkey': '[2008,7,1]', 'connection_timeout': 60000, 'inclusive_end': 'false'}, server=172.17.0.8 Key None: expected value 1008, actual: 1008 [2018-01-25 21:30:40,719] - [viewquerytests:2143] INFO - DDoc=test_view-e8b4520; View={"test_view-e8b4520": {"map": "function (doc) { if(doc.job_title !== undefined) emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] ); }", "reduce": "_count"}};: query={'startkey': '[2008,0,1]', 'endkey': '[2008,7,1]', 'inclusive_end': 'true'}: Generate results [2018-01-25 21:30:40,720] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-4e17ebd/_view/test_view-4e17ebd?startkey=%5B2008%2C0%2C1%5D&endkey=%5B2008%2C7%2C1%5D&connection_timeout=60000&inclusive_end=false [2018-01-25 21:30:40,720] - [viewquerytests:2143] INFO - DDoc=test_view-52950e3; View={"test_view-52950e3": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^UI \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};: query={'startkey': '[2008,0,1]', 'endkey': '[2008,7,1]', 'inclusive_end': 'true'}: Generate results [2018-01-25 21:30:40,720] - [viewquerytests:2143] INFO - DDoc=test_view-edcbe72; View={"test_view-edcbe72": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^System \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};: query={'startkey': '[2008,0,1]', 'endkey': '[2008,7,1]', 'inclusive_end': 'true'}: Generate results [2018-01-25 21:30:40,756] - [task:2724] INFO - view test_view-4e17ebd, query {'startkey': '[2008,0,1]', 'endkey': '[2008,7,1]', 'connection_timeout': 60000, 'inclusive_end': 'false'}: expected- 1008, actual -1008 [2018-01-25 21:30:40,757] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-3aef3b0/_view/test_view-3aef3b0?startkey=%5B2008%2C0%2C1%5D&endkey=%5B2008%2C7%2C1%5D&connection_timeout=60000&inclusive_end=false [2018-01-25 21:30:40,782] - [task:2724] INFO - view test_view-3aef3b0, query {'startkey': '[2008,0,1]', 'endkey': '[2008,7,1]', 'connection_timeout': 60000, 'inclusive_end': 'false'}: expected- 336, actual -336 [2018-01-25 21:30:40,782] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-4205ca3/_view/test_view-4205ca3?startkey=%5B2008%2C0%2C1%5D&endkey=%5B2008%2C7%2C1%5D&connection_timeout=60000&inclusive_end=false [2018-01-25 21:30:40,800] - [task:2724] INFO - view test_view-4205ca3, query {'startkey': '[2008,0,1]', 'endkey': '[2008,7,1]', 'connection_timeout': 60000, 'inclusive_end': 'false'}: expected- 336, actual -336 [2018-01-25 21:30:40,800] - [viewquerytests:2143] INFO - DDoc=test_view-3aef3b0; View={"test_view-3aef3b0": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^Senior \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};: query={'startkey': '[2008,0,1]', 'endkey': '[2008,7,1]', 'inclusive_end': 'true'}: Generate results [2018-01-25 21:30:40,801] - [viewquerytests:2143] INFO - DDoc=test_view-4e17ebd; View={"test_view-4e17ebd": {"map": "function (doc) { if(doc.job_title !== undefined) emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] ); }"}};: query={'startkey': '[2008,0,1]', 'endkey': '[2008,7,1]', 'inclusive_end': 'true'}: Generate results [2018-01-25 21:30:40,801] - [viewquerytests:2143] INFO - DDoc=test_view-4205ca3; View={"test_view-4205ca3": {"map": "function (doc, meta) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^System \"); if(doc.job_title.match(myregexp)) { emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};: query={'startkey': '[2008,0,1]', 'endkey': '[2008,7,1]', 'inclusive_end': 'true'}: Generate results [2018-01-25 21:30:40,971] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:41,124] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:41,622] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:41,623] - [viewquerytests:2159] INFO - DDoc=test_view-e8b4520; View={"test_view-e8b4520": {"map": "function (doc) { if(doc.job_title !== undefined) emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] ); }", "reduce": "_count"}};:{'startkey': '[2008,0,1]', 'endkey': '[2008,7,1]', 'inclusive_end': 'true'}: Generated results contains 1 items [2018-01-25 21:30:41,626] - [viewquerytests:2159] INFO - DDoc=test_view-52950e3; View={"test_view-52950e3": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^UI \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};:{'startkey': '[2008,0,1]', 'endkey': '[2008,7,1]', 'inclusive_end': 'true'}: Generated results contains 338 items [2018-01-25 21:30:41,630] - [viewquerytests:2159] INFO - DDoc=test_view-edcbe72; View={"test_view-edcbe72": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^System \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};:{'startkey': '[2008,0,1]', 'endkey': '[2008,7,1]', 'inclusive_end': 'true'}: Generated results contains 338 items [2018-01-25 21:30:41,848] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:42,084] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:42,438] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:42,447] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-e8b4520/_view/test_view-e8b4520?startkey=%5B2008%2C0%2C1%5D&endkey=%5B2008%2C7%2C1%5D&connection_timeout=60000&inclusive_end=true [2018-01-25 21:30:42,449] - [viewquerytests:2159] INFO - DDoc=test_view-4205ca3; View={"test_view-4205ca3": {"map": "function (doc, meta) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^System \"); if(doc.job_title.match(myregexp)) { emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};:{'startkey': '[2008,0,1]', 'endkey': '[2008,7,1]', 'inclusive_end': 'true'}: Generated results contains 338 items [2018-01-25 21:30:42,450] - [viewquerytests:2159] INFO - DDoc=test_view-3aef3b0; View={"test_view-3aef3b0": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^Senior \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};:{'startkey': '[2008,0,1]', 'endkey': '[2008,7,1]', 'inclusive_end': 'true'}: Generated results contains 338 items [2018-01-25 21:30:42,452] - [viewquerytests:2159] INFO - DDoc=test_view-4e17ebd; View={"test_view-4e17ebd": {"map": "function (doc) { if(doc.job_title !== undefined) emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] ); }"}};:{'startkey': '[2008,0,1]', 'endkey': '[2008,7,1]', 'inclusive_end': 'true'}: Generated results contains 1014 items [2018-01-25 21:30:42,499] - [task:2724] INFO - view test_view-e8b4520, query {'startkey': '[2008,0,1]', 'endkey': '[2008,7,1]', 'connection_timeout': 60000, 'inclusive_end': 'true'}: expected- 1, actual -1 [2018-01-25 21:30:42,500] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-edcbe72/_view/test_view-edcbe72?startkey=%5B2008%2C0%2C1%5D&endkey=%5B2008%2C7%2C1%5D&connection_timeout=60000&inclusive_end=true [2018-01-25 21:30:42,560] - [task:2724] INFO - view test_view-edcbe72, query {'startkey': '[2008,0,1]', 'endkey': '[2008,7,1]', 'connection_timeout': 60000, 'inclusive_end': 'true'}: expected- 338, actual -338 [2018-01-25 21:30:42,560] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-52950e3/_view/test_view-52950e3?startkey=%5B2008%2C0%2C1%5D&endkey=%5B2008%2C7%2C1%5D&connection_timeout=60000&inclusive_end=true [2018-01-25 21:30:42,635] - [task:2724] INFO - view test_view-52950e3, query {'startkey': '[2008,0,1]', 'endkey': '[2008,7,1]', 'connection_timeout': 60000, 'inclusive_end': 'true'}: expected- 338, actual -338 [2018-01-25 21:30:42,635] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'startkey': '[2008,0,1]', 'endkey': '[2008,7,1]', 'connection_timeout': 60000, 'inclusive_end': 'true'}, server=172.17.0.8 Key None: expected value 1014, actual: 1014 [2018-01-25 21:30:42,636] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'startkey': '[2008,0,1]', 'endkey': '[2008,7,1]', 'connection_timeout': 60000, 'inclusive_end': 'true'}, server=172.17.0.8 Key None: expected value 1014, actual: 1014 [2018-01-25 21:30:42,637] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-4205ca3/_view/test_view-4205ca3?startkey=%5B2008%2C0%2C1%5D&endkey=%5B2008%2C7%2C1%5D&connection_timeout=60000&inclusive_end=true [2018-01-25 21:30:42,637] - [viewquerytests:2143] INFO - DDoc=test_view-edcbe72; View={"test_view-edcbe72": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^System \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};: query={'startkey': '[2008,7,1]', 'endkey': '[2008,1,1]', 'descending': 'true', 'inclusive_end': 'false'}: Generate results [2018-01-25 21:30:42,638] - [viewquerytests:2143] INFO - DDoc=test_view-52950e3; View={"test_view-52950e3": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^UI \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};: query={'startkey': '[2008,7,1]', 'endkey': '[2008,1,1]', 'descending': 'true', 'inclusive_end': 'false'}: Generate results [2018-01-25 21:30:42,640] - [viewquerytests:2143] INFO - DDoc=test_view-e8b4520; View={"test_view-e8b4520": {"map": "function (doc) { if(doc.job_title !== undefined) emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] ); }", "reduce": "_count"}};: query={'startkey': '[2008,7,1]', 'endkey': '[2008,1,1]', 'descending': 'true', 'inclusive_end': 'false'}: Generate results [2018-01-25 21:30:42,680] - [task:2724] INFO - view test_view-4205ca3, query {'startkey': '[2008,0,1]', 'endkey': '[2008,7,1]', 'connection_timeout': 60000, 'inclusive_end': 'true'}: expected- 338, actual -338 [2018-01-25 21:30:42,680] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-4e17ebd/_view/test_view-4e17ebd?startkey=%5B2008%2C0%2C1%5D&endkey=%5B2008%2C7%2C1%5D&connection_timeout=60000&inclusive_end=true [2018-01-25 21:30:42,765] - [task:2724] INFO - view test_view-4e17ebd, query {'startkey': '[2008,0,1]', 'endkey': '[2008,7,1]', 'connection_timeout': 60000, 'inclusive_end': 'true'}: expected- 1014, actual -1014 [2018-01-25 21:30:42,765] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-3aef3b0/_view/test_view-3aef3b0?startkey=%5B2008%2C0%2C1%5D&endkey=%5B2008%2C7%2C1%5D&connection_timeout=60000&inclusive_end=true [2018-01-25 21:30:42,797] - [task:2724] INFO - view test_view-3aef3b0, query {'startkey': '[2008,0,1]', 'endkey': '[2008,7,1]', 'connection_timeout': 60000, 'inclusive_end': 'true'}: expected- 338, actual -338 [2018-01-25 21:30:42,798] - [viewquerytests:2143] INFO - DDoc=test_view-4e17ebd; View={"test_view-4e17ebd": {"map": "function (doc) { if(doc.job_title !== undefined) emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] ); }"}};: query={'startkey': '[2008,7,1]', 'endkey': '[2008,1,1]', 'descending': 'true', 'inclusive_end': 'false'}: Generate results [2018-01-25 21:30:42,799] - [viewquerytests:2143] INFO - DDoc=test_view-4205ca3; View={"test_view-4205ca3": {"map": "function (doc, meta) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^System \"); if(doc.job_title.match(myregexp)) { emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};: query={'startkey': '[2008,7,1]', 'endkey': '[2008,1,1]', 'descending': 'true', 'inclusive_end': 'false'}: Generate results [2018-01-25 21:30:42,799] - [viewquerytests:2143] INFO - DDoc=test_view-3aef3b0; View={"test_view-3aef3b0": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^Senior \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};: query={'startkey': '[2008,7,1]', 'endkey': '[2008,1,1]', 'descending': 'true', 'inclusive_end': 'false'}: Generate results [2018-01-25 21:30:43,462] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:43,814] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:44,414] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:44,416] - [viewquerytests:2159] INFO - DDoc=test_view-52950e3; View={"test_view-52950e3": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^UI \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};:{'startkey': '[2008,7,1]', 'endkey': '[2008,1,1]', 'descending': 'true', 'inclusive_end': 'false'}: Generated results contains 336 items [2018-01-25 21:30:44,417] - [viewquerytests:2159] INFO - DDoc=test_view-edcbe72; View={"test_view-edcbe72": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^System \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};:{'startkey': '[2008,7,1]', 'endkey': '[2008,1,1]', 'descending': 'true', 'inclusive_end': 'false'}: Generated results contains 336 items [2018-01-25 21:30:44,418] - [viewquerytests:2159] INFO - DDoc=test_view-e8b4520; View={"test_view-e8b4520": {"map": "function (doc) { if(doc.job_title !== undefined) emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] ); }", "reduce": "_count"}};:{'startkey': '[2008,7,1]', 'endkey': '[2008,1,1]', 'descending': 'true', 'inclusive_end': 'false'}: Generated results contains 1 items [2018-01-25 21:30:44,768] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:44,973] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:45,190] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:45,191] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-52950e3/_view/test_view-52950e3?startkey=%5B2008%2C7%2C1%5D&endkey=%5B2008%2C1%2C1%5D&descending=true&connection_timeout=60000&inclusive_end=false [2018-01-25 21:30:45,192] - [viewquerytests:2159] INFO - DDoc=test_view-4e17ebd; View={"test_view-4e17ebd": {"map": "function (doc) { if(doc.job_title !== undefined) emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] ); }"}};:{'startkey': '[2008,7,1]', 'endkey': '[2008,1,1]', 'descending': 'true', 'inclusive_end': 'false'}: Generated results contains 1008 items [2018-01-25 21:30:45,192] - [viewquerytests:2159] INFO - DDoc=test_view-4205ca3; View={"test_view-4205ca3": {"map": "function (doc, meta) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^System \"); if(doc.job_title.match(myregexp)) { emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};:{'startkey': '[2008,7,1]', 'endkey': '[2008,1,1]', 'descending': 'true', 'inclusive_end': 'false'}: Generated results contains 336 items [2018-01-25 21:30:45,192] - [viewquerytests:2159] INFO - DDoc=test_view-3aef3b0; View={"test_view-3aef3b0": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^Senior \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};:{'startkey': '[2008,7,1]', 'endkey': '[2008,1,1]', 'descending': 'true', 'inclusive_end': 'false'}: Generated results contains 336 items [2018-01-25 21:30:45,242] - [task:2724] INFO - view test_view-52950e3, query {'startkey': '[2008,7,1]', 'endkey': '[2008,1,1]', 'descending': 'true', 'connection_timeout': 60000, 'inclusive_end': 'false'}: expected- 336, actual -336 [2018-01-25 21:30:45,244] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-e8b4520/_view/test_view-e8b4520?startkey=%5B2008%2C7%2C1%5D&endkey=%5B2008%2C1%2C1%5D&descending=true&connection_timeout=60000&inclusive_end=false [2018-01-25 21:30:45,272] - [task:2724] INFO - view test_view-e8b4520, query {'startkey': '[2008,7,1]', 'endkey': '[2008,1,1]', 'descending': 'true', 'connection_timeout': 60000, 'inclusive_end': 'false'}: expected- 1, actual -1 [2018-01-25 21:30:45,273] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-edcbe72/_view/test_view-edcbe72?startkey=%5B2008%2C7%2C1%5D&endkey=%5B2008%2C1%2C1%5D&descending=true&connection_timeout=60000&inclusive_end=false [2018-01-25 21:30:45,327] - [task:2724] INFO - view test_view-edcbe72, query {'startkey': '[2008,7,1]', 'endkey': '[2008,1,1]', 'descending': 'true', 'connection_timeout': 60000, 'inclusive_end': 'false'}: expected- 336, actual -336 [2018-01-25 21:30:45,328] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'startkey': '[2008,7,1]', 'endkey': '[2008,1,1]', 'descending': 'true', 'connection_timeout': 60000, 'inclusive_end': 'false'}, server=172.17.0.8 Key None: expected value 1008, actual: 1008 [2018-01-25 21:30:45,328] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'startkey': '[2008,7,1]', 'endkey': '[2008,1,1]', 'descending': 'true', 'connection_timeout': 60000, 'inclusive_end': 'false'}, server=172.17.0.8 Key None: expected value 1008, actual: 1008 [2018-01-25 21:30:45,329] - [viewquerytests:2143] INFO - DDoc=test_view-e8b4520; View={"test_view-e8b4520": {"map": "function (doc) { if(doc.job_title !== undefined) emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] ); }", "reduce": "_count"}};: query={'startkey': '[2008,7,1]', 'endkey': '[2008,1,1]', 'descending': 'true', 'inclusive_end': 'true'}: Generate results [2018-01-25 21:30:45,329] - [viewquerytests:2143] INFO - DDoc=test_view-edcbe72; View={"test_view-edcbe72": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^System \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};: query={'startkey': '[2008,7,1]', 'endkey': '[2008,1,1]', 'descending': 'true', 'inclusive_end': 'true'}: Generate results [2018-01-25 21:30:45,329] - [viewquerytests:2143] INFO - DDoc=test_view-52950e3; View={"test_view-52950e3": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^UI \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};: query={'startkey': '[2008,7,1]', 'endkey': '[2008,1,1]', 'descending': 'true', 'inclusive_end': 'true'}: Generate results [2018-01-25 21:30:45,330] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-4e17ebd/_view/test_view-4e17ebd?startkey=%5B2008%2C7%2C1%5D&endkey=%5B2008%2C1%2C1%5D&descending=true&connection_timeout=60000&inclusive_end=false [2018-01-25 21:30:45,412] - [task:2724] INFO - view test_view-4e17ebd, query {'startkey': '[2008,7,1]', 'endkey': '[2008,1,1]', 'descending': 'true', 'connection_timeout': 60000, 'inclusive_end': 'false'}: expected- 1008, actual -1008 [2018-01-25 21:30:45,413] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-3aef3b0/_view/test_view-3aef3b0?startkey=%5B2008%2C7%2C1%5D&endkey=%5B2008%2C1%2C1%5D&descending=true&connection_timeout=60000&inclusive_end=false [2018-01-25 21:30:45,446] - [task:2724] INFO - view test_view-3aef3b0, query {'startkey': '[2008,7,1]', 'endkey': '[2008,1,1]', 'descending': 'true', 'connection_timeout': 60000, 'inclusive_end': 'false'}: expected- 336, actual -336 [2018-01-25 21:30:45,447] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-4205ca3/_view/test_view-4205ca3?startkey=%5B2008%2C7%2C1%5D&endkey=%5B2008%2C1%2C1%5D&descending=true&connection_timeout=60000&inclusive_end=false [2018-01-25 21:30:45,489] - [task:2724] INFO - view test_view-4205ca3, query {'startkey': '[2008,7,1]', 'endkey': '[2008,1,1]', 'descending': 'true', 'connection_timeout': 60000, 'inclusive_end': 'false'}: expected- 336, actual -336 [2018-01-25 21:30:45,490] - [viewquerytests:2143] INFO - DDoc=test_view-4e17ebd; View={"test_view-4e17ebd": {"map": "function (doc) { if(doc.job_title !== undefined) emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] ); }"}};: query={'startkey': '[2008,7,1]', 'endkey': '[2008,1,1]', 'descending': 'true', 'inclusive_end': 'true'}: Generate results [2018-01-25 21:30:45,490] - [viewquerytests:2143] INFO - DDoc=test_view-3aef3b0; View={"test_view-3aef3b0": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^Senior \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};: query={'startkey': '[2008,7,1]', 'endkey': '[2008,1,1]', 'descending': 'true', 'inclusive_end': 'true'}: Generate results [2018-01-25 21:30:45,491] - [viewquerytests:2143] INFO - DDoc=test_view-4205ca3; View={"test_view-4205ca3": {"map": "function (doc, meta) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^System \"); if(doc.job_title.match(myregexp)) { emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};: query={'startkey': '[2008,7,1]', 'endkey': '[2008,1,1]', 'descending': 'true', 'inclusive_end': 'true'}: Generate results [2018-01-25 21:30:45,738] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:45,954] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:46,143] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:46,144] - [viewquerytests:2159] INFO - DDoc=test_view-e8b4520; View={"test_view-e8b4520": {"map": "function (doc) { if(doc.job_title !== undefined) emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] ); }", "reduce": "_count"}};:{'startkey': '[2008,7,1]', 'endkey': '[2008,1,1]', 'descending': 'true', 'inclusive_end': 'true'}: Generated results contains 1 items [2018-01-25 21:30:46,144] - [viewquerytests:2159] INFO - DDoc=test_view-52950e3; View={"test_view-52950e3": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^UI \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};:{'startkey': '[2008,7,1]', 'endkey': '[2008,1,1]', 'descending': 'true', 'inclusive_end': 'true'}: Generated results contains 338 items [2018-01-25 21:30:46,145] - [viewquerytests:2159] INFO - DDoc=test_view-edcbe72; View={"test_view-edcbe72": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^System \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};:{'startkey': '[2008,7,1]', 'endkey': '[2008,1,1]', 'descending': 'true', 'inclusive_end': 'true'}: Generated results contains 338 items [2018-01-25 21:30:46,384] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:46,658] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:46,873] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:46,874] - [viewquerytests:2159] INFO - DDoc=test_view-4e17ebd; View={"test_view-4e17ebd": {"map": "function (doc) { if(doc.job_title !== undefined) emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] ); }"}};:{'startkey': '[2008,7,1]', 'endkey': '[2008,1,1]', 'descending': 'true', 'inclusive_end': 'true'}: Generated results contains 1014 items [2018-01-25 21:30:46,875] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-e8b4520/_view/test_view-e8b4520?startkey=%5B2008%2C7%2C1%5D&endkey=%5B2008%2C1%2C1%5D&descending=true&connection_timeout=60000&inclusive_end=true [2018-01-25 21:30:46,875] - [viewquerytests:2159] INFO - DDoc=test_view-4205ca3; View={"test_view-4205ca3": {"map": "function (doc, meta) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^System \"); if(doc.job_title.match(myregexp)) { emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};:{'startkey': '[2008,7,1]', 'endkey': '[2008,1,1]', 'descending': 'true', 'inclusive_end': 'true'}: Generated results contains 338 items [2018-01-25 21:30:46,875] - [viewquerytests:2159] INFO - DDoc=test_view-3aef3b0; View={"test_view-3aef3b0": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^Senior \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};:{'startkey': '[2008,7,1]', 'endkey': '[2008,1,1]', 'descending': 'true', 'inclusive_end': 'true'}: Generated results contains 338 items [2018-01-25 21:30:46,908] - [task:2724] INFO - view test_view-e8b4520, query {'startkey': '[2008,7,1]', 'endkey': '[2008,1,1]', 'descending': 'true', 'connection_timeout': 60000, 'inclusive_end': 'true'}: expected- 1, actual -1 [2018-01-25 21:30:46,909] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-edcbe72/_view/test_view-edcbe72?startkey=%5B2008%2C7%2C1%5D&endkey=%5B2008%2C1%2C1%5D&descending=true&connection_timeout=60000&inclusive_end=true [2018-01-25 21:30:46,991] - [task:2724] INFO - view test_view-edcbe72, query {'startkey': '[2008,7,1]', 'endkey': '[2008,1,1]', 'descending': 'true', 'connection_timeout': 60000, 'inclusive_end': 'true'}: expected- 338, actual -338 [2018-01-25 21:30:46,992] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-52950e3/_view/test_view-52950e3?startkey=%5B2008%2C7%2C1%5D&endkey=%5B2008%2C1%2C1%5D&descending=true&connection_timeout=60000&inclusive_end=true [2018-01-25 21:30:47,033] - [task:2724] INFO - view test_view-52950e3, query {'startkey': '[2008,7,1]', 'endkey': '[2008,1,1]', 'descending': 'true', 'connection_timeout': 60000, 'inclusive_end': 'true'}: expected- 338, actual -338 [2018-01-25 21:30:47,035] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'startkey': '[2008,7,1]', 'endkey': '[2008,1,1]', 'descending': 'true', 'connection_timeout': 60000, 'inclusive_end': 'true'}, server=172.17.0.8 Key None: expected value 1014, actual: 1014 [2018-01-25 21:30:47,035] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'startkey': '[2008,7,1]', 'endkey': '[2008,1,1]', 'descending': 'true', 'connection_timeout': 60000, 'inclusive_end': 'true'}, server=172.17.0.8 Key None: expected value 1014, actual: 1014 [2018-01-25 21:30:47,038] - [viewquerytests:2143] INFO - DDoc=test_view-e8b4520; View={"test_view-e8b4520": {"map": "function (doc) { if(doc.job_title !== undefined) emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] ); }", "reduce": "_count"}};: query={'startkey': '[2008,1,1]', 'endkey': '[2008,7,1]', 'descending': 'false', 'inclusive_end': 'false'}: Generate results [2018-01-25 21:30:47,039] - [viewquerytests:2143] INFO - DDoc=test_view-edcbe72; View={"test_view-edcbe72": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^System \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};: query={'startkey': '[2008,1,1]', 'endkey': '[2008,7,1]', 'descending': 'false', 'inclusive_end': 'false'}: Generate results [2018-01-25 21:30:47,040] - [viewquerytests:2143] INFO - DDoc=test_view-52950e3; View={"test_view-52950e3": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^UI \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};: query={'startkey': '[2008,1,1]', 'endkey': '[2008,7,1]', 'descending': 'false', 'inclusive_end': 'false'}: Generate results [2018-01-25 21:30:47,042] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-4e17ebd/_view/test_view-4e17ebd?startkey=%5B2008%2C7%2C1%5D&endkey=%5B2008%2C1%2C1%5D&descending=true&connection_timeout=60000&inclusive_end=true [2018-01-25 21:30:47,116] - [task:2724] INFO - view test_view-4e17ebd, query {'startkey': '[2008,7,1]', 'endkey': '[2008,1,1]', 'descending': 'true', 'connection_timeout': 60000, 'inclusive_end': 'true'}: expected- 1014, actual -1014 [2018-01-25 21:30:47,117] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-3aef3b0/_view/test_view-3aef3b0?startkey=%5B2008%2C7%2C1%5D&endkey=%5B2008%2C1%2C1%5D&descending=true&connection_timeout=60000&inclusive_end=true [2018-01-25 21:30:47,164] - [task:2724] INFO - view test_view-3aef3b0, query {'startkey': '[2008,7,1]', 'endkey': '[2008,1,1]', 'descending': 'true', 'connection_timeout': 60000, 'inclusive_end': 'true'}: expected- 338, actual -338 [2018-01-25 21:30:47,165] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-4205ca3/_view/test_view-4205ca3?startkey=%5B2008%2C7%2C1%5D&endkey=%5B2008%2C1%2C1%5D&descending=true&connection_timeout=60000&inclusive_end=true [2018-01-25 21:30:47,233] - [task:2724] INFO - view test_view-4205ca3, query {'startkey': '[2008,7,1]', 'endkey': '[2008,1,1]', 'descending': 'true', 'connection_timeout': 60000, 'inclusive_end': 'true'}: expected- 338, actual -338 [2018-01-25 21:30:47,237] - [viewquerytests:2143] INFO - DDoc=test_view-3aef3b0; View={"test_view-3aef3b0": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^Senior \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};: query={'startkey': '[2008,1,1]', 'endkey': '[2008,7,1]', 'descending': 'false', 'inclusive_end': 'false'}: Generate results [2018-01-25 21:30:47,237] - [viewquerytests:2143] INFO - DDoc=test_view-4e17ebd; View={"test_view-4e17ebd": {"map": "function (doc) { if(doc.job_title !== undefined) emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] ); }"}};: query={'startkey': '[2008,1,1]', 'endkey': '[2008,7,1]', 'descending': 'false', 'inclusive_end': 'false'}: Generate results [2018-01-25 21:30:47,238] - [viewquerytests:2143] INFO - DDoc=test_view-4205ca3; View={"test_view-4205ca3": {"map": "function (doc, meta) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^System \"); if(doc.job_title.match(myregexp)) { emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};: query={'startkey': '[2008,1,1]', 'endkey': '[2008,7,1]', 'descending': 'false', 'inclusive_end': 'false'}: Generate results [2018-01-25 21:30:47,672] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:48,015] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:48,243] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:48,244] - [viewquerytests:2159] INFO - DDoc=test_view-edcbe72; View={"test_view-edcbe72": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^System \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};:{'startkey': '[2008,1,1]', 'endkey': '[2008,7,1]', 'descending': 'false', 'inclusive_end': 'false'}: Generated results contains 336 items [2018-01-25 21:30:48,245] - [viewquerytests:2159] INFO - DDoc=test_view-52950e3; View={"test_view-52950e3": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^UI \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};:{'startkey': '[2008,1,1]', 'endkey': '[2008,7,1]', 'descending': 'false', 'inclusive_end': 'false'}: Generated results contains 336 items [2018-01-25 21:30:48,245] - [viewquerytests:2159] INFO - DDoc=test_view-e8b4520; View={"test_view-e8b4520": {"map": "function (doc) { if(doc.job_title !== undefined) emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] ); }", "reduce": "_count"}};:{'startkey': '[2008,1,1]', 'endkey': '[2008,7,1]', 'descending': 'false', 'inclusive_end': 'false'}: Generated results contains 1 items [2018-01-25 21:30:48,474] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:48,651] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:48,822] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:48,823] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-52950e3/_view/test_view-52950e3?startkey=%5B2008%2C1%2C1%5D&endkey=%5B2008%2C7%2C1%5D&descending=false&connection_timeout=60000&inclusive_end=false [2018-01-25 21:30:48,823] - [viewquerytests:2159] INFO - DDoc=test_view-3aef3b0; View={"test_view-3aef3b0": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^Senior \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};:{'startkey': '[2008,1,1]', 'endkey': '[2008,7,1]', 'descending': 'false', 'inclusive_end': 'false'}: Generated results contains 336 items [2018-01-25 21:30:48,824] - [viewquerytests:2159] INFO - DDoc=test_view-4e17ebd; View={"test_view-4e17ebd": {"map": "function (doc) { if(doc.job_title !== undefined) emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] ); }"}};:{'startkey': '[2008,1,1]', 'endkey': '[2008,7,1]', 'descending': 'false', 'inclusive_end': 'false'}: Generated results contains 1008 items [2018-01-25 21:30:48,824] - [viewquerytests:2159] INFO - DDoc=test_view-4205ca3; View={"test_view-4205ca3": {"map": "function (doc, meta) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^System \"); if(doc.job_title.match(myregexp)) { emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};:{'startkey': '[2008,1,1]', 'endkey': '[2008,7,1]', 'descending': 'false', 'inclusive_end': 'false'}: Generated results contains 336 items [2018-01-25 21:30:48,849] - [task:2724] INFO - view test_view-52950e3, query {'startkey': '[2008,1,1]', 'endkey': '[2008,7,1]', 'descending': 'false', 'connection_timeout': 60000, 'inclusive_end': 'false'}: expected- 336, actual -336 [2018-01-25 21:30:48,849] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-edcbe72/_view/test_view-edcbe72?startkey=%5B2008%2C1%2C1%5D&endkey=%5B2008%2C7%2C1%5D&descending=false&connection_timeout=60000&inclusive_end=false [2018-01-25 21:30:48,869] - [task:2724] INFO - view test_view-edcbe72, query {'startkey': '[2008,1,1]', 'endkey': '[2008,7,1]', 'descending': 'false', 'connection_timeout': 60000, 'inclusive_end': 'false'}: expected- 336, actual -336 [2018-01-25 21:30:48,869] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-e8b4520/_view/test_view-e8b4520?startkey=%5B2008%2C1%2C1%5D&endkey=%5B2008%2C7%2C1%5D&descending=false&connection_timeout=60000&inclusive_end=false [2018-01-25 21:30:48,890] - [task:2724] INFO - view test_view-e8b4520, query {'startkey': '[2008,1,1]', 'endkey': '[2008,7,1]', 'descending': 'false', 'connection_timeout': 60000, 'inclusive_end': 'false'}: expected- 1, actual -1 [2018-01-25 21:30:48,890] - [viewquerytests:2143] INFO - DDoc=test_view-52950e3; View={"test_view-52950e3": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^UI \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};: query={'startkey': '[2008,1,1]', 'endkey': '[2008,7,1]', 'descending': 'false', 'inclusive_end': 'true'}: Generate results [2018-01-25 21:30:48,891] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'startkey': '[2008,1,1]', 'endkey': '[2008,7,1]', 'descending': 'false', 'connection_timeout': 60000, 'inclusive_end': 'false'}, server=172.17.0.8 Key None: expected value 1008, actual: 1008 [2018-01-25 21:30:48,891] - [viewquerytests:2143] INFO - DDoc=test_view-edcbe72; View={"test_view-edcbe72": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^System \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};: query={'startkey': '[2008,1,1]', 'endkey': '[2008,7,1]', 'descending': 'false', 'inclusive_end': 'true'}: Generate results [2018-01-25 21:30:48,891] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'startkey': '[2008,1,1]', 'endkey': '[2008,7,1]', 'descending': 'false', 'connection_timeout': 60000, 'inclusive_end': 'false'}, server=172.17.0.8 Key None: expected value 1008, actual: 1008 [2018-01-25 21:30:48,892] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-3aef3b0/_view/test_view-3aef3b0?startkey=%5B2008%2C1%2C1%5D&endkey=%5B2008%2C7%2C1%5D&descending=false&connection_timeout=60000&inclusive_end=false [2018-01-25 21:30:48,892] - [viewquerytests:2143] INFO - DDoc=test_view-e8b4520; View={"test_view-e8b4520": {"map": "function (doc) { if(doc.job_title !== undefined) emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] ); }", "reduce": "_count"}};: query={'startkey': '[2008,1,1]', 'endkey': '[2008,7,1]', 'descending': 'false', 'inclusive_end': 'true'}: Generate results [2018-01-25 21:30:48,931] - [task:2724] INFO - view test_view-3aef3b0, query {'startkey': '[2008,1,1]', 'endkey': '[2008,7,1]', 'descending': 'false', 'connection_timeout': 60000, 'inclusive_end': 'false'}: expected- 336, actual -336 [2018-01-25 21:30:48,932] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-4e17ebd/_view/test_view-4e17ebd?startkey=%5B2008%2C1%2C1%5D&endkey=%5B2008%2C7%2C1%5D&descending=false&connection_timeout=60000&inclusive_end=false [2018-01-25 21:30:48,986] - [task:2724] INFO - view test_view-4e17ebd, query {'startkey': '[2008,1,1]', 'endkey': '[2008,7,1]', 'descending': 'false', 'connection_timeout': 60000, 'inclusive_end': 'false'}: expected- 1008, actual -1008 [2018-01-25 21:30:48,987] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-4205ca3/_view/test_view-4205ca3?startkey=%5B2008%2C1%2C1%5D&endkey=%5B2008%2C7%2C1%5D&descending=false&connection_timeout=60000&inclusive_end=false [2018-01-25 21:30:49,023] - [task:2724] INFO - view test_view-4205ca3, query {'startkey': '[2008,1,1]', 'endkey': '[2008,7,1]', 'descending': 'false', 'connection_timeout': 60000, 'inclusive_end': 'false'}: expected- 336, actual -336 [2018-01-25 21:30:49,023] - [viewquerytests:2143] INFO - DDoc=test_view-3aef3b0; View={"test_view-3aef3b0": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^Senior \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};: query={'startkey': '[2008,1,1]', 'endkey': '[2008,7,1]', 'descending': 'false', 'inclusive_end': 'true'}: Generate results [2018-01-25 21:30:49,024] - [viewquerytests:2143] INFO - DDoc=test_view-4e17ebd; View={"test_view-4e17ebd": {"map": "function (doc) { if(doc.job_title !== undefined) emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] ); }"}};: query={'startkey': '[2008,1,1]', 'endkey': '[2008,7,1]', 'descending': 'false', 'inclusive_end': 'true'}: Generate results [2018-01-25 21:30:49,024] - [viewquerytests:2143] INFO - DDoc=test_view-4205ca3; View={"test_view-4205ca3": {"map": "function (doc, meta) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^System \"); if(doc.job_title.match(myregexp)) { emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};: query={'startkey': '[2008,1,1]', 'endkey': '[2008,7,1]', 'descending': 'false', 'inclusive_end': 'true'}: Generate results [2018-01-25 21:30:49,221] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:49,402] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:49,554] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:49,554] - [viewquerytests:2159] INFO - DDoc=test_view-52950e3; View={"test_view-52950e3": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^UI \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};:{'startkey': '[2008,1,1]', 'endkey': '[2008,7,1]', 'descending': 'false', 'inclusive_end': 'true'}: Generated results contains 338 items [2018-01-25 21:30:49,555] - [viewquerytests:2159] INFO - DDoc=test_view-edcbe72; View={"test_view-edcbe72": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^System \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};:{'startkey': '[2008,1,1]', 'endkey': '[2008,7,1]', 'descending': 'false', 'inclusive_end': 'true'}: Generated results contains 338 items [2018-01-25 21:30:49,556] - [viewquerytests:2159] INFO - DDoc=test_view-e8b4520; View={"test_view-e8b4520": {"map": "function (doc) { if(doc.job_title !== undefined) emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] ); }", "reduce": "_count"}};:{'startkey': '[2008,1,1]', 'endkey': '[2008,7,1]', 'descending': 'false', 'inclusive_end': 'true'}: Generated results contains 1 items [2018-01-25 21:30:49,741] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:49,909] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:50,077] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:50,078] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-52950e3/_view/test_view-52950e3?startkey=%5B2008%2C1%2C1%5D&endkey=%5B2008%2C7%2C1%5D&descending=false&connection_timeout=60000&inclusive_end=true [2018-01-25 21:30:50,078] - [viewquerytests:2159] INFO - DDoc=test_view-3aef3b0; View={"test_view-3aef3b0": {"map": "function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^Senior \"); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};:{'startkey': '[2008,1,1]', 'endkey': '[2008,7,1]', 'descending': 'false', 'inclusive_end': 'true'}: Generated results contains 338 items [2018-01-25 21:30:50,079] - [viewquerytests:2159] INFO - DDoc=test_view-4e17ebd; View={"test_view-4e17ebd": {"map": "function (doc) { if(doc.job_title !== undefined) emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] ); }"}};:{'startkey': '[2008,1,1]', 'endkey': '[2008,7,1]', 'descending': 'false', 'inclusive_end': 'true'}: Generated results contains 1014 items [2018-01-25 21:30:50,079] - [viewquerytests:2159] INFO - DDoc=test_view-4205ca3; View={"test_view-4205ca3": {"map": "function (doc, meta) { if(doc.job_title !== undefined) { var myregexp = new RegExp(\"^System \"); if(doc.job_title.match(myregexp)) { emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}"}};:{'startkey': '[2008,1,1]', 'endkey': '[2008,7,1]', 'descending': 'false', 'inclusive_end': 'true'}: Generated results contains 338 items [2018-01-25 21:30:50,103] - [task:2724] INFO - view test_view-52950e3, query {'startkey': '[2008,1,1]', 'endkey': '[2008,7,1]', 'descending': 'false', 'connection_timeout': 60000, 'inclusive_end': 'true'}: expected- 338, actual -338 [2018-01-25 21:30:50,104] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-edcbe72/_view/test_view-edcbe72?startkey=%5B2008%2C1%2C1%5D&endkey=%5B2008%2C7%2C1%5D&descending=false&connection_timeout=60000&inclusive_end=true [2018-01-25 21:30:50,126] - [task:2724] INFO - view test_view-edcbe72, query {'startkey': '[2008,1,1]', 'endkey': '[2008,7,1]', 'descending': 'false', 'connection_timeout': 60000, 'inclusive_end': 'true'}: expected- 338, actual -338 [2018-01-25 21:30:50,126] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-e8b4520/_view/test_view-e8b4520?startkey=%5B2008%2C1%2C1%5D&endkey=%5B2008%2C7%2C1%5D&descending=false&connection_timeout=60000&inclusive_end=true [2018-01-25 21:30:50,141] - [task:2724] INFO - view test_view-e8b4520, query {'startkey': '[2008,1,1]', 'endkey': '[2008,7,1]', 'descending': 'false', 'connection_timeout': 60000, 'inclusive_end': 'true'}: expected- 1, actual -1 [2018-01-25 21:30:50,142] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'startkey': '[2008,1,1]', 'endkey': '[2008,7,1]', 'descending': 'false', 'connection_timeout': 60000, 'inclusive_end': 'true'}, server=172.17.0.8 Key None: expected value 1014, actual: 1014 [2018-01-25 21:30:50,142] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'startkey': '[2008,1,1]', 'endkey': '[2008,7,1]', 'descending': 'false', 'connection_timeout': 60000, 'inclusive_end': 'true'}, server=172.17.0.8 Key None: expected value 1014, actual: 1014 [2018-01-25 21:30:50,143] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-3aef3b0/_view/test_view-3aef3b0?startkey=%5B2008%2C1%2C1%5D&endkey=%5B2008%2C7%2C1%5D&descending=false&connection_timeout=60000&inclusive_end=true [2018-01-25 21:30:50,144] - [viewquerytests:2143] INFO - DDoc=test_view-e8b4520; View={"test_view-e8b4520": {"map": "function (doc) { if(doc.job_title !== undefined) emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] ); }", "reduce": "_count"}};: query={'group': 'true'}: Generate results [2018-01-25 21:30:50,175] - [viewquerytests:2047] INFO - Current amount of threads 4 [2018-01-25 21:30:50,225] - [task:2724] INFO - view test_view-3aef3b0, query {'startkey': '[2008,1,1]', 'endkey': '[2008,7,1]', 'descending': 'false', 'connection_timeout': 60000, 'inclusive_end': 'true'}: expected- 338, actual -338 [2018-01-25 21:30:50,228] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-4205ca3/_view/test_view-4205ca3?startkey=%5B2008%2C1%2C1%5D&endkey=%5B2008%2C7%2C1%5D&descending=false&connection_timeout=60000&inclusive_end=true [2018-01-25 21:30:50,295] - [task:2724] INFO - view test_view-4205ca3, query {'startkey': '[2008,1,1]', 'endkey': '[2008,7,1]', 'descending': 'false', 'connection_timeout': 60000, 'inclusive_end': 'true'}: expected- 338, actual -338 [2018-01-25 21:30:50,301] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-4e17ebd/_view/test_view-4e17ebd?startkey=%5B2008%2C1%2C1%5D&endkey=%5B2008%2C7%2C1%5D&descending=false&connection_timeout=60000&inclusive_end=true [2018-01-25 21:30:50,513] - [task:2724] INFO - view test_view-4e17ebd, query {'startkey': '[2008,1,1]', 'endkey': '[2008,7,1]', 'descending': 'false', 'connection_timeout': 60000, 'inclusive_end': 'true'}: expected- 1014, actual -1014 [2018-01-25 21:30:50,550] - [viewquerytests:2047] INFO - Current amount of threads 1 [2018-01-25 21:30:50,805] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:50,806] - [viewquerytests:2159] INFO - DDoc=test_view-e8b4520; View={"test_view-e8b4520": {"map": "function (doc) { if(doc.job_title !== undefined) emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] ); }", "reduce": "_count"}};:{'group': 'true'}: Generated results contains 336 items [2018-01-25 21:30:51,807] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-e8b4520/_view/test_view-e8b4520?connection_timeout=60000&group=true [2018-01-25 21:30:51,872] - [task:2724] INFO - view test_view-e8b4520, query {'connection_timeout': 60000, 'group': 'true'}: expected- 336, actual -336 [2018-01-25 21:30:51,873] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 1]: expected value 6, actual: 6 [2018-01-25 21:30:51,873] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 1]: expected value 6, actual: 6 [2018-01-25 21:30:51,874] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 2]: expected value 6, actual: 6 [2018-01-25 21:30:51,874] - [viewquerytests:2143] INFO - DDoc=test_view-e8b4520; View={"test_view-e8b4520": {"map": "function (doc) { if(doc.job_title !== undefined) emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] ); }", "reduce": "_count"}};: query={'group_level': '1'}: Generate results [2018-01-25 21:30:51,875] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 2]: expected value 6, actual: 6 [2018-01-25 21:30:51,878] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 3]: expected value 6, actual: 6 [2018-01-25 21:30:51,879] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 3]: expected value 6, actual: 6 [2018-01-25 21:30:51,880] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 4]: expected value 6, actual: 6 [2018-01-25 21:30:51,880] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 4]: expected value 6, actual: 6 [2018-01-25 21:30:51,881] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 5]: expected value 6, actual: 6 [2018-01-25 21:30:51,881] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 5]: expected value 6, actual: 6 [2018-01-25 21:30:51,881] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 6]: expected value 6, actual: 6 [2018-01-25 21:30:51,882] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 6]: expected value 6, actual: 6 [2018-01-25 21:30:51,882] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 7]: expected value 6, actual: 6 [2018-01-25 21:30:51,882] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 7]: expected value 6, actual: 6 [2018-01-25 21:30:51,883] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 8]: expected value 6, actual: 6 [2018-01-25 21:30:51,883] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 8]: expected value 6, actual: 6 [2018-01-25 21:30:51,883] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 9]: expected value 6, actual: 6 [2018-01-25 21:30:51,884] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 9]: expected value 6, actual: 6 [2018-01-25 21:30:51,884] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 10]: expected value 6, actual: 6 [2018-01-25 21:30:51,885] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 10]: expected value 6, actual: 6 [2018-01-25 21:30:51,885] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 11]: expected value 6, actual: 6 [2018-01-25 21:30:51,886] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 11]: expected value 6, actual: 6 [2018-01-25 21:30:51,886] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 12]: expected value 6, actual: 6 [2018-01-25 21:30:51,886] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 12]: expected value 6, actual: 6 [2018-01-25 21:30:51,886] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 13]: expected value 6, actual: 6 [2018-01-25 21:30:51,886] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 13]: expected value 6, actual: 6 [2018-01-25 21:30:51,887] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 14]: expected value 6, actual: 6 [2018-01-25 21:30:51,887] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 14]: expected value 6, actual: 6 [2018-01-25 21:30:51,887] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 15]: expected value 6, actual: 6 [2018-01-25 21:30:51,888] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 15]: expected value 6, actual: 6 [2018-01-25 21:30:51,888] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 16]: expected value 6, actual: 6 [2018-01-25 21:30:51,888] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 16]: expected value 6, actual: 6 [2018-01-25 21:30:51,888] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 17]: expected value 6, actual: 6 [2018-01-25 21:30:51,889] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 17]: expected value 6, actual: 6 [2018-01-25 21:30:51,889] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 18]: expected value 6, actual: 6 [2018-01-25 21:30:51,889] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 18]: expected value 6, actual: 6 [2018-01-25 21:30:51,889] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 19]: expected value 6, actual: 6 [2018-01-25 21:30:51,889] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 19]: expected value 6, actual: 6 [2018-01-25 21:30:51,889] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 20]: expected value 6, actual: 6 [2018-01-25 21:30:51,889] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 20]: expected value 6, actual: 6 [2018-01-25 21:30:51,890] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 21]: expected value 6, actual: 6 [2018-01-25 21:30:51,890] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 21]: expected value 6, actual: 6 [2018-01-25 21:30:51,890] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 22]: expected value 6, actual: 6 [2018-01-25 21:30:51,890] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 22]: expected value 6, actual: 6 [2018-01-25 21:30:51,891] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 23]: expected value 6, actual: 6 [2018-01-25 21:30:51,891] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 23]: expected value 6, actual: 6 [2018-01-25 21:30:51,891] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 24]: expected value 6, actual: 6 [2018-01-25 21:30:51,891] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 24]: expected value 6, actual: 6 [2018-01-25 21:30:51,892] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 25]: expected value 6, actual: 6 [2018-01-25 21:30:51,892] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 25]: expected value 6, actual: 6 [2018-01-25 21:30:51,892] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 26]: expected value 6, actual: 6 [2018-01-25 21:30:51,892] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 26]: expected value 6, actual: 6 [2018-01-25 21:30:51,893] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 27]: expected value 6, actual: 6 [2018-01-25 21:30:51,893] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 27]: expected value 6, actual: 6 [2018-01-25 21:30:51,893] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 28]: expected value 6, actual: 6 [2018-01-25 21:30:51,893] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 1, 28]: expected value 6, actual: 6 [2018-01-25 21:30:51,893] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 1]: expected value 6, actual: 6 [2018-01-25 21:30:51,893] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 1]: expected value 6, actual: 6 [2018-01-25 21:30:51,894] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 2]: expected value 6, actual: 6 [2018-01-25 21:30:51,894] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 2]: expected value 6, actual: 6 [2018-01-25 21:30:51,894] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 3]: expected value 6, actual: 6 [2018-01-25 21:30:51,894] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 3]: expected value 6, actual: 6 [2018-01-25 21:30:51,895] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 4]: expected value 6, actual: 6 [2018-01-25 21:30:51,895] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 4]: expected value 6, actual: 6 [2018-01-25 21:30:51,895] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 5]: expected value 6, actual: 6 [2018-01-25 21:30:51,896] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 5]: expected value 6, actual: 6 [2018-01-25 21:30:51,896] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 6]: expected value 6, actual: 6 [2018-01-25 21:30:51,896] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 6]: expected value 6, actual: 6 [2018-01-25 21:30:51,897] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 7]: expected value 6, actual: 6 [2018-01-25 21:30:51,897] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 7]: expected value 6, actual: 6 [2018-01-25 21:30:51,897] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 8]: expected value 6, actual: 6 [2018-01-25 21:30:51,898] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 8]: expected value 6, actual: 6 [2018-01-25 21:30:51,898] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 9]: expected value 6, actual: 6 [2018-01-25 21:30:51,898] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 9]: expected value 6, actual: 6 [2018-01-25 21:30:51,899] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 10]: expected value 6, actual: 6 [2018-01-25 21:30:51,899] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 10]: expected value 6, actual: 6 [2018-01-25 21:30:51,899] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 11]: expected value 6, actual: 6 [2018-01-25 21:30:51,899] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 11]: expected value 6, actual: 6 [2018-01-25 21:30:51,899] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 12]: expected value 6, actual: 6 [2018-01-25 21:30:51,900] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 12]: expected value 6, actual: 6 [2018-01-25 21:30:51,900] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 13]: expected value 6, actual: 6 [2018-01-25 21:30:51,900] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 13]: expected value 6, actual: 6 [2018-01-25 21:30:51,901] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 14]: expected value 6, actual: 6 [2018-01-25 21:30:51,901] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 14]: expected value 6, actual: 6 [2018-01-25 21:30:51,901] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 15]: expected value 6, actual: 6 [2018-01-25 21:30:51,902] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 15]: expected value 6, actual: 6 [2018-01-25 21:30:51,902] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 16]: expected value 6, actual: 6 [2018-01-25 21:30:51,902] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 16]: expected value 6, actual: 6 [2018-01-25 21:30:51,903] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 17]: expected value 6, actual: 6 [2018-01-25 21:30:51,903] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 17]: expected value 6, actual: 6 [2018-01-25 21:30:51,904] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 18]: expected value 6, actual: 6 [2018-01-25 21:30:51,904] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 18]: expected value 6, actual: 6 [2018-01-25 21:30:51,904] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 19]: expected value 6, actual: 6 [2018-01-25 21:30:51,904] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 19]: expected value 6, actual: 6 [2018-01-25 21:30:51,904] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 20]: expected value 6, actual: 6 [2018-01-25 21:30:51,905] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 20]: expected value 6, actual: 6 [2018-01-25 21:30:51,905] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 21]: expected value 6, actual: 6 [2018-01-25 21:30:51,905] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 21]: expected value 6, actual: 6 [2018-01-25 21:30:51,905] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 22]: expected value 6, actual: 6 [2018-01-25 21:30:51,905] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 22]: expected value 6, actual: 6 [2018-01-25 21:30:51,906] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 23]: expected value 6, actual: 6 [2018-01-25 21:30:51,906] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 23]: expected value 6, actual: 6 [2018-01-25 21:30:51,906] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 24]: expected value 6, actual: 6 [2018-01-25 21:30:51,906] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 24]: expected value 6, actual: 6 [2018-01-25 21:30:51,907] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 25]: expected value 6, actual: 6 [2018-01-25 21:30:51,907] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 25]: expected value 6, actual: 6 [2018-01-25 21:30:51,907] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 26]: expected value 6, actual: 6 [2018-01-25 21:30:51,907] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 26]: expected value 6, actual: 6 [2018-01-25 21:30:51,907] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 27]: expected value 6, actual: 6 [2018-01-25 21:30:51,908] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 27]: expected value 6, actual: 6 [2018-01-25 21:30:51,908] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 28]: expected value 6, actual: 6 [2018-01-25 21:30:51,908] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 2, 28]: expected value 6, actual: 6 [2018-01-25 21:30:51,908] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 1]: expected value 6, actual: 6 [2018-01-25 21:30:51,908] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 1]: expected value 6, actual: 6 [2018-01-25 21:30:51,909] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 2]: expected value 6, actual: 6 [2018-01-25 21:30:51,909] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 2]: expected value 6, actual: 6 [2018-01-25 21:30:51,909] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 3]: expected value 6, actual: 6 [2018-01-25 21:30:51,909] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 3]: expected value 6, actual: 6 [2018-01-25 21:30:51,909] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 4]: expected value 6, actual: 6 [2018-01-25 21:30:51,910] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 4]: expected value 6, actual: 6 [2018-01-25 21:30:51,910] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 5]: expected value 6, actual: 6 [2018-01-25 21:30:51,911] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 5]: expected value 6, actual: 6 [2018-01-25 21:30:51,911] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 6]: expected value 6, actual: 6 [2018-01-25 21:30:51,911] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 6]: expected value 6, actual: 6 [2018-01-25 21:30:51,912] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 7]: expected value 6, actual: 6 [2018-01-25 21:30:51,912] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 7]: expected value 6, actual: 6 [2018-01-25 21:30:51,912] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 8]: expected value 6, actual: 6 [2018-01-25 21:30:51,912] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 8]: expected value 6, actual: 6 [2018-01-25 21:30:51,913] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 9]: expected value 6, actual: 6 [2018-01-25 21:30:51,913] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 9]: expected value 6, actual: 6 [2018-01-25 21:30:51,914] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 10]: expected value 6, actual: 6 [2018-01-25 21:30:51,914] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 10]: expected value 6, actual: 6 [2018-01-25 21:30:51,917] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 11]: expected value 6, actual: 6 [2018-01-25 21:30:51,917] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 11]: expected value 6, actual: 6 [2018-01-25 21:30:51,918] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 12]: expected value 6, actual: 6 [2018-01-25 21:30:51,919] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 12]: expected value 6, actual: 6 [2018-01-25 21:30:51,919] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 13]: expected value 6, actual: 6 [2018-01-25 21:30:51,919] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 13]: expected value 6, actual: 6 [2018-01-25 21:30:51,920] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 14]: expected value 6, actual: 6 [2018-01-25 21:30:51,920] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 14]: expected value 6, actual: 6 [2018-01-25 21:30:51,921] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 15]: expected value 6, actual: 6 [2018-01-25 21:30:51,921] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 15]: expected value 6, actual: 6 [2018-01-25 21:30:51,921] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 16]: expected value 6, actual: 6 [2018-01-25 21:30:51,922] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 16]: expected value 6, actual: 6 [2018-01-25 21:30:51,922] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 17]: expected value 6, actual: 6 [2018-01-25 21:30:51,922] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 17]: expected value 6, actual: 6 [2018-01-25 21:30:51,923] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 18]: expected value 6, actual: 6 [2018-01-25 21:30:51,923] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 18]: expected value 6, actual: 6 [2018-01-25 21:30:51,924] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 19]: expected value 6, actual: 6 [2018-01-25 21:30:51,924] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 19]: expected value 6, actual: 6 [2018-01-25 21:30:51,925] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 20]: expected value 6, actual: 6 [2018-01-25 21:30:51,925] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 20]: expected value 6, actual: 6 [2018-01-25 21:30:51,925] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 21]: expected value 6, actual: 6 [2018-01-25 21:30:51,926] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 21]: expected value 6, actual: 6 [2018-01-25 21:30:51,926] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 22]: expected value 6, actual: 6 [2018-01-25 21:30:51,926] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 22]: expected value 6, actual: 6 [2018-01-25 21:30:51,927] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 23]: expected value 6, actual: 6 [2018-01-25 21:30:51,927] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 23]: expected value 6, actual: 6 [2018-01-25 21:30:51,928] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 24]: expected value 6, actual: 6 [2018-01-25 21:30:51,928] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 24]: expected value 6, actual: 6 [2018-01-25 21:30:51,928] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 25]: expected value 6, actual: 6 [2018-01-25 21:30:51,928] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 25]: expected value 6, actual: 6 [2018-01-25 21:30:51,929] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 26]: expected value 6, actual: 6 [2018-01-25 21:30:51,929] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 26]: expected value 6, actual: 6 [2018-01-25 21:30:51,930] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 27]: expected value 6, actual: 6 [2018-01-25 21:30:51,930] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 27]: expected value 6, actual: 6 [2018-01-25 21:30:51,931] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 28]: expected value 6, actual: 6 [2018-01-25 21:30:51,931] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 3, 28]: expected value 6, actual: 6 [2018-01-25 21:30:51,931] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 1]: expected value 6, actual: 6 [2018-01-25 21:30:51,931] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 1]: expected value 6, actual: 6 [2018-01-25 21:30:51,932] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 2]: expected value 6, actual: 6 [2018-01-25 21:30:51,932] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 2]: expected value 6, actual: 6 [2018-01-25 21:30:51,932] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 3]: expected value 6, actual: 6 [2018-01-25 21:30:51,932] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 3]: expected value 6, actual: 6 [2018-01-25 21:30:51,933] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 4]: expected value 6, actual: 6 [2018-01-25 21:30:51,933] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 4]: expected value 6, actual: 6 [2018-01-25 21:30:51,934] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 5]: expected value 6, actual: 6 [2018-01-25 21:30:51,934] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 5]: expected value 6, actual: 6 [2018-01-25 21:30:51,935] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 6]: expected value 6, actual: 6 [2018-01-25 21:30:51,935] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 6]: expected value 6, actual: 6 [2018-01-25 21:30:51,936] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 7]: expected value 6, actual: 6 [2018-01-25 21:30:51,936] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 7]: expected value 6, actual: 6 [2018-01-25 21:30:51,937] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 8]: expected value 6, actual: 6 [2018-01-25 21:30:51,937] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 8]: expected value 6, actual: 6 [2018-01-25 21:30:51,939] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 9]: expected value 6, actual: 6 [2018-01-25 21:30:51,939] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 9]: expected value 6, actual: 6 [2018-01-25 21:30:51,940] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 10]: expected value 6, actual: 6 [2018-01-25 21:30:51,940] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 10]: expected value 6, actual: 6 [2018-01-25 21:30:51,941] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 11]: expected value 6, actual: 6 [2018-01-25 21:30:51,941] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 11]: expected value 6, actual: 6 [2018-01-25 21:30:51,942] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 12]: expected value 6, actual: 6 [2018-01-25 21:30:51,942] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 12]: expected value 6, actual: 6 [2018-01-25 21:30:51,942] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 13]: expected value 6, actual: 6 [2018-01-25 21:30:51,942] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 13]: expected value 6, actual: 6 [2018-01-25 21:30:51,943] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 14]: expected value 6, actual: 6 [2018-01-25 21:30:51,943] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 14]: expected value 6, actual: 6 [2018-01-25 21:30:51,943] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 15]: expected value 6, actual: 6 [2018-01-25 21:30:51,943] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 15]: expected value 6, actual: 6 [2018-01-25 21:30:51,944] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 16]: expected value 6, actual: 6 [2018-01-25 21:30:51,944] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 16]: expected value 6, actual: 6 [2018-01-25 21:30:51,944] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 17]: expected value 6, actual: 6 [2018-01-25 21:30:51,944] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 17]: expected value 6, actual: 6 [2018-01-25 21:30:51,945] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 18]: expected value 6, actual: 6 [2018-01-25 21:30:51,945] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 18]: expected value 6, actual: 6 [2018-01-25 21:30:51,945] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 19]: expected value 6, actual: 6 [2018-01-25 21:30:51,946] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 19]: expected value 6, actual: 6 [2018-01-25 21:30:51,946] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 20]: expected value 6, actual: 6 [2018-01-25 21:30:51,947] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 20]: expected value 6, actual: 6 [2018-01-25 21:30:51,947] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 21]: expected value 6, actual: 6 [2018-01-25 21:30:51,948] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 21]: expected value 6, actual: 6 [2018-01-25 21:30:51,948] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 22]: expected value 6, actual: 6 [2018-01-25 21:30:51,948] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 22]: expected value 6, actual: 6 [2018-01-25 21:30:51,949] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 23]: expected value 6, actual: 6 [2018-01-25 21:30:51,949] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 23]: expected value 6, actual: 6 [2018-01-25 21:30:51,949] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 24]: expected value 6, actual: 6 [2018-01-25 21:30:51,950] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 24]: expected value 6, actual: 6 [2018-01-25 21:30:51,950] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 25]: expected value 6, actual: 6 [2018-01-25 21:30:51,950] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 25]: expected value 6, actual: 6 [2018-01-25 21:30:51,950] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 26]: expected value 6, actual: 6 [2018-01-25 21:30:51,950] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 26]: expected value 6, actual: 6 [2018-01-25 21:30:51,951] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 27]: expected value 6, actual: 6 [2018-01-25 21:30:51,951] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 27]: expected value 6, actual: 6 [2018-01-25 21:30:51,951] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 28]: expected value 6, actual: 6 [2018-01-25 21:30:51,951] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 4, 28]: expected value 6, actual: 6 [2018-01-25 21:30:51,952] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 1]: expected value 6, actual: 6 [2018-01-25 21:30:51,952] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 1]: expected value 6, actual: 6 [2018-01-25 21:30:51,952] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 2]: expected value 6, actual: 6 [2018-01-25 21:30:51,953] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 2]: expected value 6, actual: 6 [2018-01-25 21:30:51,953] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 3]: expected value 6, actual: 6 [2018-01-25 21:30:51,953] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 3]: expected value 6, actual: 6 [2018-01-25 21:30:51,954] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 4]: expected value 6, actual: 6 [2018-01-25 21:30:51,954] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 4]: expected value 6, actual: 6 [2018-01-25 21:30:51,954] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 5]: expected value 6, actual: 6 [2018-01-25 21:30:51,955] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 5]: expected value 6, actual: 6 [2018-01-25 21:30:51,955] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 6]: expected value 6, actual: 6 [2018-01-25 21:30:51,956] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 6]: expected value 6, actual: 6 [2018-01-25 21:30:51,956] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 7]: expected value 6, actual: 6 [2018-01-25 21:30:51,956] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 7]: expected value 6, actual: 6 [2018-01-25 21:30:51,957] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 8]: expected value 6, actual: 6 [2018-01-25 21:30:51,957] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 8]: expected value 6, actual: 6 [2018-01-25 21:30:51,957] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 9]: expected value 6, actual: 6 [2018-01-25 21:30:51,957] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 9]: expected value 6, actual: 6 [2018-01-25 21:30:51,958] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 10]: expected value 6, actual: 6 [2018-01-25 21:30:51,958] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 10]: expected value 6, actual: 6 [2018-01-25 21:30:51,958] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 11]: expected value 6, actual: 6 [2018-01-25 21:30:51,959] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 11]: expected value 6, actual: 6 [2018-01-25 21:30:51,959] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 12]: expected value 6, actual: 6 [2018-01-25 21:30:51,959] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 12]: expected value 6, actual: 6 [2018-01-25 21:30:51,960] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 13]: expected value 6, actual: 6 [2018-01-25 21:30:51,960] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 13]: expected value 6, actual: 6 [2018-01-25 21:30:51,961] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 14]: expected value 6, actual: 6 [2018-01-25 21:30:51,961] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 14]: expected value 6, actual: 6 [2018-01-25 21:30:51,961] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 15]: expected value 6, actual: 6 [2018-01-25 21:30:51,962] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 15]: expected value 6, actual: 6 [2018-01-25 21:30:51,962] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 16]: expected value 6, actual: 6 [2018-01-25 21:30:51,963] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 16]: expected value 6, actual: 6 [2018-01-25 21:30:51,964] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 17]: expected value 6, actual: 6 [2018-01-25 21:30:51,965] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 17]: expected value 6, actual: 6 [2018-01-25 21:30:51,965] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 18]: expected value 6, actual: 6 [2018-01-25 21:30:51,966] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 18]: expected value 6, actual: 6 [2018-01-25 21:30:51,966] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 19]: expected value 6, actual: 6 [2018-01-25 21:30:51,966] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 19]: expected value 6, actual: 6 [2018-01-25 21:30:51,967] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 20]: expected value 6, actual: 6 [2018-01-25 21:30:51,967] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 20]: expected value 6, actual: 6 [2018-01-25 21:30:51,967] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 21]: expected value 6, actual: 6 [2018-01-25 21:30:51,967] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 21]: expected value 6, actual: 6 [2018-01-25 21:30:51,967] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 22]: expected value 6, actual: 6 [2018-01-25 21:30:51,967] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 22]: expected value 6, actual: 6 [2018-01-25 21:30:51,967] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 23]: expected value 6, actual: 6 [2018-01-25 21:30:51,968] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 23]: expected value 6, actual: 6 [2018-01-25 21:30:51,968] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 24]: expected value 6, actual: 6 [2018-01-25 21:30:51,968] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 24]: expected value 6, actual: 6 [2018-01-25 21:30:51,968] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 25]: expected value 6, actual: 6 [2018-01-25 21:30:51,969] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 25]: expected value 6, actual: 6 [2018-01-25 21:30:51,969] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 26]: expected value 6, actual: 6 [2018-01-25 21:30:51,969] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 26]: expected value 6, actual: 6 [2018-01-25 21:30:51,969] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 27]: expected value 6, actual: 6 [2018-01-25 21:30:51,969] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 27]: expected value 6, actual: 6 [2018-01-25 21:30:51,969] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 28]: expected value 6, actual: 6 [2018-01-25 21:30:51,970] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 5, 28]: expected value 6, actual: 6 [2018-01-25 21:30:51,970] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 1]: expected value 6, actual: 6 [2018-01-25 21:30:51,970] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 1]: expected value 6, actual: 6 [2018-01-25 21:30:51,970] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 2]: expected value 6, actual: 6 [2018-01-25 21:30:51,970] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 2]: expected value 6, actual: 6 [2018-01-25 21:30:51,971] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 3]: expected value 6, actual: 6 [2018-01-25 21:30:51,971] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 3]: expected value 6, actual: 6 [2018-01-25 21:30:51,971] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 4]: expected value 6, actual: 6 [2018-01-25 21:30:51,971] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 4]: expected value 6, actual: 6 [2018-01-25 21:30:51,971] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 5]: expected value 6, actual: 6 [2018-01-25 21:30:51,972] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 5]: expected value 6, actual: 6 [2018-01-25 21:30:51,972] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 6]: expected value 6, actual: 6 [2018-01-25 21:30:51,972] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 6]: expected value 6, actual: 6 [2018-01-25 21:30:51,972] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 7]: expected value 6, actual: 6 [2018-01-25 21:30:51,972] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 7]: expected value 6, actual: 6 [2018-01-25 21:30:51,973] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 8]: expected value 6, actual: 6 [2018-01-25 21:30:51,973] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 8]: expected value 6, actual: 6 [2018-01-25 21:30:51,973] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 9]: expected value 6, actual: 6 [2018-01-25 21:30:51,973] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 9]: expected value 6, actual: 6 [2018-01-25 21:30:51,974] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 10]: expected value 6, actual: 6 [2018-01-25 21:30:51,974] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 10]: expected value 6, actual: 6 [2018-01-25 21:30:51,975] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 11]: expected value 6, actual: 6 [2018-01-25 21:30:51,975] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 11]: expected value 6, actual: 6 [2018-01-25 21:30:51,976] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 12]: expected value 6, actual: 6 [2018-01-25 21:30:51,976] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 12]: expected value 6, actual: 6 [2018-01-25 21:30:51,976] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 13]: expected value 6, actual: 6 [2018-01-25 21:30:51,977] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 13]: expected value 6, actual: 6 [2018-01-25 21:30:51,977] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 14]: expected value 6, actual: 6 [2018-01-25 21:30:51,977] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 14]: expected value 6, actual: 6 [2018-01-25 21:30:51,978] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 15]: expected value 6, actual: 6 [2018-01-25 21:30:51,978] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 15]: expected value 6, actual: 6 [2018-01-25 21:30:51,978] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 16]: expected value 6, actual: 6 [2018-01-25 21:30:51,979] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 16]: expected value 6, actual: 6 [2018-01-25 21:30:51,979] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 17]: expected value 6, actual: 6 [2018-01-25 21:30:51,979] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 17]: expected value 6, actual: 6 [2018-01-25 21:30:51,979] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 18]: expected value 6, actual: 6 [2018-01-25 21:30:51,979] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 18]: expected value 6, actual: 6 [2018-01-25 21:30:51,980] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 19]: expected value 6, actual: 6 [2018-01-25 21:30:51,980] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 19]: expected value 6, actual: 6 [2018-01-25 21:30:51,980] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 20]: expected value 6, actual: 6 [2018-01-25 21:30:51,981] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 20]: expected value 6, actual: 6 [2018-01-25 21:30:51,981] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 21]: expected value 6, actual: 6 [2018-01-25 21:30:51,981] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 21]: expected value 6, actual: 6 [2018-01-25 21:30:51,982] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 22]: expected value 6, actual: 6 [2018-01-25 21:30:51,982] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 22]: expected value 6, actual: 6 [2018-01-25 21:30:51,982] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 23]: expected value 6, actual: 6 [2018-01-25 21:30:51,982] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 23]: expected value 6, actual: 6 [2018-01-25 21:30:51,983] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 24]: expected value 6, actual: 6 [2018-01-25 21:30:51,983] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 24]: expected value 6, actual: 6 [2018-01-25 21:30:51,984] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 25]: expected value 6, actual: 6 [2018-01-25 21:30:51,984] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 25]: expected value 6, actual: 6 [2018-01-25 21:30:51,984] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 26]: expected value 6, actual: 6 [2018-01-25 21:30:51,984] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 26]: expected value 6, actual: 6 [2018-01-25 21:30:51,984] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 27]: expected value 6, actual: 6 [2018-01-25 21:30:51,985] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 27]: expected value 6, actual: 6 [2018-01-25 21:30:51,985] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 28]: expected value 6, actual: 6 [2018-01-25 21:30:51,985] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 6, 28]: expected value 6, actual: 6 [2018-01-25 21:30:51,985] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 1]: expected value 6, actual: 6 [2018-01-25 21:30:51,986] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 1]: expected value 6, actual: 6 [2018-01-25 21:30:51,986] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 2]: expected value 6, actual: 6 [2018-01-25 21:30:51,986] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 2]: expected value 6, actual: 6 [2018-01-25 21:30:51,986] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 3]: expected value 6, actual: 6 [2018-01-25 21:30:51,987] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 3]: expected value 6, actual: 6 [2018-01-25 21:30:51,987] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 4]: expected value 6, actual: 6 [2018-01-25 21:30:51,987] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 4]: expected value 6, actual: 6 [2018-01-25 21:30:51,988] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 5]: expected value 6, actual: 6 [2018-01-25 21:30:51,988] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 5]: expected value 6, actual: 6 [2018-01-25 21:30:51,988] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 6]: expected value 6, actual: 6 [2018-01-25 21:30:51,988] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 6]: expected value 6, actual: 6 [2018-01-25 21:30:51,988] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 7]: expected value 6, actual: 6 [2018-01-25 21:30:51,989] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 7]: expected value 6, actual: 6 [2018-01-25 21:30:51,989] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 8]: expected value 6, actual: 6 [2018-01-25 21:30:51,989] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 8]: expected value 6, actual: 6 [2018-01-25 21:30:51,989] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 9]: expected value 6, actual: 6 [2018-01-25 21:30:51,989] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 9]: expected value 6, actual: 6 [2018-01-25 21:30:51,989] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 10]: expected value 6, actual: 6 [2018-01-25 21:30:51,990] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 10]: expected value 6, actual: 6 [2018-01-25 21:30:51,990] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 11]: expected value 6, actual: 6 [2018-01-25 21:30:51,990] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 11]: expected value 6, actual: 6 [2018-01-25 21:30:51,990] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 12]: expected value 6, actual: 6 [2018-01-25 21:30:51,990] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 12]: expected value 6, actual: 6 [2018-01-25 21:30:51,991] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 13]: expected value 6, actual: 6 [2018-01-25 21:30:51,991] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 13]: expected value 6, actual: 6 [2018-01-25 21:30:51,991] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 14]: expected value 6, actual: 6 [2018-01-25 21:30:51,992] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 14]: expected value 6, actual: 6 [2018-01-25 21:30:51,993] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 15]: expected value 6, actual: 6 [2018-01-25 21:30:51,993] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 15]: expected value 6, actual: 6 [2018-01-25 21:30:51,993] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 16]: expected value 6, actual: 6 [2018-01-25 21:30:51,994] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 16]: expected value 6, actual: 6 [2018-01-25 21:30:51,994] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 17]: expected value 6, actual: 6 [2018-01-25 21:30:51,994] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 17]: expected value 6, actual: 6 [2018-01-25 21:30:51,995] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 18]: expected value 6, actual: 6 [2018-01-25 21:30:51,995] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 18]: expected value 6, actual: 6 [2018-01-25 21:30:51,996] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 19]: expected value 6, actual: 6 [2018-01-25 21:30:51,996] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 19]: expected value 6, actual: 6 [2018-01-25 21:30:51,996] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 20]: expected value 6, actual: 6 [2018-01-25 21:30:51,997] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 20]: expected value 6, actual: 6 [2018-01-25 21:30:51,997] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 21]: expected value 6, actual: 6 [2018-01-25 21:30:51,997] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 21]: expected value 6, actual: 6 [2018-01-25 21:30:51,998] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 22]: expected value 6, actual: 6 [2018-01-25 21:30:51,998] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 22]: expected value 6, actual: 6 [2018-01-25 21:30:51,999] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 23]: expected value 6, actual: 6 [2018-01-25 21:30:52,000] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 23]: expected value 6, actual: 6 [2018-01-25 21:30:52,001] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 24]: expected value 6, actual: 6 [2018-01-25 21:30:52,001] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 24]: expected value 6, actual: 6 [2018-01-25 21:30:52,002] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 25]: expected value 6, actual: 6 [2018-01-25 21:30:52,003] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 25]: expected value 6, actual: 6 [2018-01-25 21:30:52,004] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 26]: expected value 6, actual: 6 [2018-01-25 21:30:52,004] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 26]: expected value 6, actual: 6 [2018-01-25 21:30:52,004] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 27]: expected value 6, actual: 6 [2018-01-25 21:30:52,005] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 27]: expected value 6, actual: 6 [2018-01-25 21:30:52,005] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 28]: expected value 6, actual: 6 [2018-01-25 21:30:52,005] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 7, 28]: expected value 6, actual: 6 [2018-01-25 21:30:52,006] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 1]: expected value 6, actual: 6 [2018-01-25 21:30:52,006] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 1]: expected value 6, actual: 6 [2018-01-25 21:30:52,007] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 2]: expected value 6, actual: 6 [2018-01-25 21:30:52,008] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 2]: expected value 6, actual: 6 [2018-01-25 21:30:52,008] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 3]: expected value 6, actual: 6 [2018-01-25 21:30:52,009] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 3]: expected value 6, actual: 6 [2018-01-25 21:30:52,010] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 4]: expected value 6, actual: 6 [2018-01-25 21:30:52,010] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 4]: expected value 6, actual: 6 [2018-01-25 21:30:52,011] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 5]: expected value 6, actual: 6 [2018-01-25 21:30:52,011] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 5]: expected value 6, actual: 6 [2018-01-25 21:30:52,012] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 6]: expected value 6, actual: 6 [2018-01-25 21:30:52,012] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 6]: expected value 6, actual: 6 [2018-01-25 21:30:52,012] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 7]: expected value 6, actual: 6 [2018-01-25 21:30:52,012] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 7]: expected value 6, actual: 6 [2018-01-25 21:30:52,013] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 8]: expected value 6, actual: 6 [2018-01-25 21:30:52,013] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 8]: expected value 6, actual: 6 [2018-01-25 21:30:52,014] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 9]: expected value 6, actual: 6 [2018-01-25 21:30:52,014] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 9]: expected value 6, actual: 6 [2018-01-25 21:30:52,014] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 10]: expected value 6, actual: 6 [2018-01-25 21:30:52,015] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 10]: expected value 6, actual: 6 [2018-01-25 21:30:52,015] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 11]: expected value 6, actual: 6 [2018-01-25 21:30:52,016] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 11]: expected value 6, actual: 6 [2018-01-25 21:30:52,016] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 12]: expected value 6, actual: 6 [2018-01-25 21:30:52,016] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 12]: expected value 6, actual: 6 [2018-01-25 21:30:52,017] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 13]: expected value 6, actual: 6 [2018-01-25 21:30:52,017] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 13]: expected value 6, actual: 6 [2018-01-25 21:30:52,018] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 14]: expected value 6, actual: 6 [2018-01-25 21:30:52,018] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 14]: expected value 6, actual: 6 [2018-01-25 21:30:52,018] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 15]: expected value 6, actual: 6 [2018-01-25 21:30:52,018] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 15]: expected value 6, actual: 6 [2018-01-25 21:30:52,019] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 16]: expected value 6, actual: 6 [2018-01-25 21:30:52,019] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 16]: expected value 6, actual: 6 [2018-01-25 21:30:52,019] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 17]: expected value 6, actual: 6 [2018-01-25 21:30:52,019] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 17]: expected value 6, actual: 6 [2018-01-25 21:30:52,020] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 18]: expected value 6, actual: 6 [2018-01-25 21:30:52,020] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 18]: expected value 6, actual: 6 [2018-01-25 21:30:52,021] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 19]: expected value 6, actual: 6 [2018-01-25 21:30:52,021] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 19]: expected value 6, actual: 6 [2018-01-25 21:30:52,021] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 20]: expected value 6, actual: 6 [2018-01-25 21:30:52,021] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 20]: expected value 6, actual: 6 [2018-01-25 21:30:52,022] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 21]: expected value 6, actual: 6 [2018-01-25 21:30:52,022] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 21]: expected value 6, actual: 6 [2018-01-25 21:30:52,023] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 22]: expected value 6, actual: 6 [2018-01-25 21:30:52,023] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 22]: expected value 6, actual: 6 [2018-01-25 21:30:52,024] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 23]: expected value 6, actual: 6 [2018-01-25 21:30:52,024] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 23]: expected value 6, actual: 6 [2018-01-25 21:30:52,024] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 24]: expected value 6, actual: 6 [2018-01-25 21:30:52,024] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 24]: expected value 6, actual: 6 [2018-01-25 21:30:52,025] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 25]: expected value 6, actual: 6 [2018-01-25 21:30:52,025] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 25]: expected value 6, actual: 6 [2018-01-25 21:30:52,026] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 26]: expected value 6, actual: 6 [2018-01-25 21:30:52,026] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 26]: expected value 6, actual: 6 [2018-01-25 21:30:52,026] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 27]: expected value 6, actual: 6 [2018-01-25 21:30:52,026] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 27]: expected value 6, actual: 6 [2018-01-25 21:30:52,027] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 28]: expected value 6, actual: 6 [2018-01-25 21:30:52,027] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 8, 28]: expected value 6, actual: 6 [2018-01-25 21:30:52,027] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 1]: expected value 6, actual: 6 [2018-01-25 21:30:52,027] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 1]: expected value 6, actual: 6 [2018-01-25 21:30:52,028] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 2]: expected value 6, actual: 6 [2018-01-25 21:30:52,028] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 2]: expected value 6, actual: 6 [2018-01-25 21:30:52,028] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 3]: expected value 6, actual: 6 [2018-01-25 21:30:52,028] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 3]: expected value 6, actual: 6 [2018-01-25 21:30:52,029] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 4]: expected value 6, actual: 6 [2018-01-25 21:30:52,029] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 4]: expected value 6, actual: 6 [2018-01-25 21:30:52,029] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 5]: expected value 6, actual: 6 [2018-01-25 21:30:52,030] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 5]: expected value 6, actual: 6 [2018-01-25 21:30:52,030] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 6]: expected value 6, actual: 6 [2018-01-25 21:30:52,030] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 6]: expected value 6, actual: 6 [2018-01-25 21:30:52,031] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 7]: expected value 6, actual: 6 [2018-01-25 21:30:52,031] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 7]: expected value 6, actual: 6 [2018-01-25 21:30:52,031] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 8]: expected value 6, actual: 6 [2018-01-25 21:30:52,031] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 8]: expected value 6, actual: 6 [2018-01-25 21:30:52,032] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 9]: expected value 6, actual: 6 [2018-01-25 21:30:52,032] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 9]: expected value 6, actual: 6 [2018-01-25 21:30:52,033] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 10]: expected value 6, actual: 6 [2018-01-25 21:30:52,033] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 10]: expected value 6, actual: 6 [2018-01-25 21:30:52,033] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 11]: expected value 6, actual: 6 [2018-01-25 21:30:52,034] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 11]: expected value 6, actual: 6 [2018-01-25 21:30:52,034] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 12]: expected value 6, actual: 6 [2018-01-25 21:30:52,034] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 12]: expected value 6, actual: 6 [2018-01-25 21:30:52,035] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 13]: expected value 6, actual: 6 [2018-01-25 21:30:52,035] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 13]: expected value 6, actual: 6 [2018-01-25 21:30:52,035] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 14]: expected value 6, actual: 6 [2018-01-25 21:30:52,035] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 14]: expected value 6, actual: 6 [2018-01-25 21:30:52,036] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 15]: expected value 6, actual: 6 [2018-01-25 21:30:52,036] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 15]: expected value 6, actual: 6 [2018-01-25 21:30:52,036] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 16]: expected value 6, actual: 6 [2018-01-25 21:30:52,036] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 16]: expected value 6, actual: 6 [2018-01-25 21:30:52,037] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 17]: expected value 6, actual: 6 [2018-01-25 21:30:52,037] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 17]: expected value 6, actual: 6 [2018-01-25 21:30:52,037] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 18]: expected value 6, actual: 6 [2018-01-25 21:30:52,038] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 18]: expected value 6, actual: 6 [2018-01-25 21:30:52,038] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 19]: expected value 6, actual: 6 [2018-01-25 21:30:52,038] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 19]: expected value 6, actual: 6 [2018-01-25 21:30:52,039] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 20]: expected value 6, actual: 6 [2018-01-25 21:30:52,039] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 20]: expected value 6, actual: 6 [2018-01-25 21:30:52,039] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 21]: expected value 6, actual: 6 [2018-01-25 21:30:52,040] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 21]: expected value 6, actual: 6 [2018-01-25 21:30:52,040] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 22]: expected value 6, actual: 6 [2018-01-25 21:30:52,040] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 22]: expected value 6, actual: 6 [2018-01-25 21:30:52,041] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 23]: expected value 6, actual: 6 [2018-01-25 21:30:52,041] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 23]: expected value 6, actual: 6 [2018-01-25 21:30:52,041] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 24]: expected value 6, actual: 6 [2018-01-25 21:30:52,041] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 24]: expected value 6, actual: 6 [2018-01-25 21:30:52,042] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 25]: expected value 6, actual: 6 [2018-01-25 21:30:52,042] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 25]: expected value 6, actual: 6 [2018-01-25 21:30:52,042] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 26]: expected value 6, actual: 6 [2018-01-25 21:30:52,043] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 26]: expected value 6, actual: 6 [2018-01-25 21:30:52,043] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 27]: expected value 6, actual: 6 [2018-01-25 21:30:52,043] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 27]: expected value 6, actual: 6 [2018-01-25 21:30:52,044] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 28]: expected value 6, actual: 6 [2018-01-25 21:30:52,044] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 9, 28]: expected value 6, actual: 6 [2018-01-25 21:30:52,044] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 1]: expected value 6, actual: 6 [2018-01-25 21:30:52,044] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 1]: expected value 6, actual: 6 [2018-01-25 21:30:52,045] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 2]: expected value 6, actual: 6 [2018-01-25 21:30:52,045] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 2]: expected value 6, actual: 6 [2018-01-25 21:30:52,046] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 3]: expected value 6, actual: 6 [2018-01-25 21:30:52,046] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 3]: expected value 6, actual: 6 [2018-01-25 21:30:52,047] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 4]: expected value 6, actual: 6 [2018-01-25 21:30:52,047] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 4]: expected value 6, actual: 6 [2018-01-25 21:30:52,047] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 5]: expected value 6, actual: 6 [2018-01-25 21:30:52,048] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 5]: expected value 6, actual: 6 [2018-01-25 21:30:52,048] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 6]: expected value 6, actual: 6 [2018-01-25 21:30:52,049] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 6]: expected value 6, actual: 6 [2018-01-25 21:30:52,049] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 7]: expected value 6, actual: 6 [2018-01-25 21:30:52,049] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 7]: expected value 6, actual: 6 [2018-01-25 21:30:52,050] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 8]: expected value 6, actual: 6 [2018-01-25 21:30:52,050] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 8]: expected value 6, actual: 6 [2018-01-25 21:30:52,051] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 9]: expected value 6, actual: 6 [2018-01-25 21:30:52,051] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 9]: expected value 6, actual: 6 [2018-01-25 21:30:52,052] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 10]: expected value 6, actual: 6 [2018-01-25 21:30:52,052] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 10]: expected value 6, actual: 6 [2018-01-25 21:30:52,053] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 11]: expected value 6, actual: 6 [2018-01-25 21:30:52,053] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 11]: expected value 6, actual: 6 [2018-01-25 21:30:52,054] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 12]: expected value 6, actual: 6 [2018-01-25 21:30:52,054] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 12]: expected value 6, actual: 6 [2018-01-25 21:30:52,055] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 13]: expected value 6, actual: 6 [2018-01-25 21:30:52,055] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 13]: expected value 6, actual: 6 [2018-01-25 21:30:52,055] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 14]: expected value 6, actual: 6 [2018-01-25 21:30:52,056] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 14]: expected value 6, actual: 6 [2018-01-25 21:30:52,056] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 15]: expected value 6, actual: 6 [2018-01-25 21:30:52,056] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 15]: expected value 6, actual: 6 [2018-01-25 21:30:52,057] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 16]: expected value 6, actual: 6 [2018-01-25 21:30:52,057] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 16]: expected value 6, actual: 6 [2018-01-25 21:30:52,057] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 17]: expected value 6, actual: 6 [2018-01-25 21:30:52,057] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 17]: expected value 6, actual: 6 [2018-01-25 21:30:52,058] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 18]: expected value 6, actual: 6 [2018-01-25 21:30:52,058] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 18]: expected value 6, actual: 6 [2018-01-25 21:30:52,058] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 19]: expected value 6, actual: 6 [2018-01-25 21:30:52,058] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 19]: expected value 6, actual: 6 [2018-01-25 21:30:52,059] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 20]: expected value 6, actual: 6 [2018-01-25 21:30:52,059] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 20]: expected value 6, actual: 6 [2018-01-25 21:30:52,059] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 21]: expected value 6, actual: 6 [2018-01-25 21:30:52,060] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 21]: expected value 6, actual: 6 [2018-01-25 21:30:52,060] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 22]: expected value 6, actual: 6 [2018-01-25 21:30:52,060] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 22]: expected value 6, actual: 6 [2018-01-25 21:30:52,061] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 23]: expected value 6, actual: 6 [2018-01-25 21:30:52,061] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 23]: expected value 6, actual: 6 [2018-01-25 21:30:52,061] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 24]: expected value 6, actual: 6 [2018-01-25 21:30:52,061] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 24]: expected value 6, actual: 6 [2018-01-25 21:30:52,062] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 25]: expected value 6, actual: 6 [2018-01-25 21:30:52,062] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 25]: expected value 6, actual: 6 [2018-01-25 21:30:52,062] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 26]: expected value 6, actual: 6 [2018-01-25 21:30:52,062] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 26]: expected value 6, actual: 6 [2018-01-25 21:30:52,063] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 27]: expected value 6, actual: 6 [2018-01-25 21:30:52,063] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 27]: expected value 6, actual: 6 [2018-01-25 21:30:52,064] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 28]: expected value 6, actual: 6 [2018-01-25 21:30:52,064] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 10, 28]: expected value 6, actual: 6 [2018-01-25 21:30:52,066] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 1]: expected value 6, actual: 6 [2018-01-25 21:30:52,066] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 1]: expected value 6, actual: 6 [2018-01-25 21:30:52,067] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 2]: expected value 6, actual: 6 [2018-01-25 21:30:52,067] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 2]: expected value 6, actual: 6 [2018-01-25 21:30:52,068] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 3]: expected value 6, actual: 6 [2018-01-25 21:30:52,068] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 3]: expected value 6, actual: 6 [2018-01-25 21:30:52,069] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 4]: expected value 6, actual: 6 [2018-01-25 21:30:52,069] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 4]: expected value 6, actual: 6 [2018-01-25 21:30:52,069] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 5]: expected value 6, actual: 6 [2018-01-25 21:30:52,070] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 5]: expected value 6, actual: 6 [2018-01-25 21:30:52,070] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 6]: expected value 6, actual: 6 [2018-01-25 21:30:52,070] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 6]: expected value 6, actual: 6 [2018-01-25 21:30:52,071] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 7]: expected value 6, actual: 6 [2018-01-25 21:30:52,071] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 7]: expected value 6, actual: 6 [2018-01-25 21:30:52,071] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 8]: expected value 6, actual: 6 [2018-01-25 21:30:52,071] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 8]: expected value 6, actual: 6 [2018-01-25 21:30:52,072] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 9]: expected value 6, actual: 6 [2018-01-25 21:30:52,072] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 9]: expected value 6, actual: 6 [2018-01-25 21:30:52,072] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 10]: expected value 6, actual: 6 [2018-01-25 21:30:52,073] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 10]: expected value 6, actual: 6 [2018-01-25 21:30:52,073] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 11]: expected value 6, actual: 6 [2018-01-25 21:30:52,074] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 11]: expected value 6, actual: 6 [2018-01-25 21:30:52,074] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 12]: expected value 6, actual: 6 [2018-01-25 21:30:52,074] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 12]: expected value 6, actual: 6 [2018-01-25 21:30:52,075] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 13]: expected value 6, actual: 6 [2018-01-25 21:30:52,075] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 13]: expected value 6, actual: 6 [2018-01-25 21:30:52,075] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 14]: expected value 6, actual: 6 [2018-01-25 21:30:52,075] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 14]: expected value 6, actual: 6 [2018-01-25 21:30:52,075] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 15]: expected value 6, actual: 6 [2018-01-25 21:30:52,075] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 15]: expected value 6, actual: 6 [2018-01-25 21:30:52,076] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 16]: expected value 6, actual: 6 [2018-01-25 21:30:52,076] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 16]: expected value 6, actual: 6 [2018-01-25 21:30:52,076] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 17]: expected value 6, actual: 6 [2018-01-25 21:30:52,076] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 17]: expected value 6, actual: 6 [2018-01-25 21:30:52,077] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 18]: expected value 6, actual: 6 [2018-01-25 21:30:52,077] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 18]: expected value 6, actual: 6 [2018-01-25 21:30:52,077] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 19]: expected value 6, actual: 6 [2018-01-25 21:30:52,077] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 19]: expected value 6, actual: 6 [2018-01-25 21:30:52,077] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 20]: expected value 6, actual: 6 [2018-01-25 21:30:52,077] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 20]: expected value 6, actual: 6 [2018-01-25 21:30:52,078] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 21]: expected value 6, actual: 6 [2018-01-25 21:30:52,078] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 21]: expected value 6, actual: 6 [2018-01-25 21:30:52,078] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 22]: expected value 6, actual: 6 [2018-01-25 21:30:52,079] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 22]: expected value 6, actual: 6 [2018-01-25 21:30:52,079] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 23]: expected value 6, actual: 6 [2018-01-25 21:30:52,079] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 23]: expected value 6, actual: 6 [2018-01-25 21:30:52,080] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 24]: expected value 6, actual: 6 [2018-01-25 21:30:52,080] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 24]: expected value 6, actual: 6 [2018-01-25 21:30:52,080] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 25]: expected value 6, actual: 6 [2018-01-25 21:30:52,080] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 25]: expected value 6, actual: 6 [2018-01-25 21:30:52,081] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 26]: expected value 6, actual: 6 [2018-01-25 21:30:52,081] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 26]: expected value 6, actual: 6 [2018-01-25 21:30:52,081] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 27]: expected value 6, actual: 6 [2018-01-25 21:30:52,081] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 27]: expected value 6, actual: 6 [2018-01-25 21:30:52,081] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 28]: expected value 6, actual: 6 [2018-01-25 21:30:52,082] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 11, 28]: expected value 6, actual: 6 [2018-01-25 21:30:52,082] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 1]: expected value 6, actual: 6 [2018-01-25 21:30:52,082] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 1]: expected value 6, actual: 6 [2018-01-25 21:30:52,083] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 2]: expected value 6, actual: 6 [2018-01-25 21:30:52,083] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 2]: expected value 6, actual: 6 [2018-01-25 21:30:52,083] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 3]: expected value 6, actual: 6 [2018-01-25 21:30:52,083] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 3]: expected value 6, actual: 6 [2018-01-25 21:30:52,083] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 4]: expected value 6, actual: 6 [2018-01-25 21:30:52,083] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 4]: expected value 6, actual: 6 [2018-01-25 21:30:52,083] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 5]: expected value 6, actual: 6 [2018-01-25 21:30:52,084] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 5]: expected value 6, actual: 6 [2018-01-25 21:30:52,084] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 6]: expected value 6, actual: 6 [2018-01-25 21:30:52,084] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 6]: expected value 6, actual: 6 [2018-01-25 21:30:52,084] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 7]: expected value 6, actual: 6 [2018-01-25 21:30:52,084] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 7]: expected value 6, actual: 6 [2018-01-25 21:30:52,085] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 8]: expected value 6, actual: 6 [2018-01-25 21:30:52,085] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 8]: expected value 6, actual: 6 [2018-01-25 21:30:52,085] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 9]: expected value 6, actual: 6 [2018-01-25 21:30:52,085] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 9]: expected value 6, actual: 6 [2018-01-25 21:30:52,085] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 10]: expected value 6, actual: 6 [2018-01-25 21:30:52,086] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 10]: expected value 6, actual: 6 [2018-01-25 21:30:52,086] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 11]: expected value 6, actual: 6 [2018-01-25 21:30:52,086] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 11]: expected value 6, actual: 6 [2018-01-25 21:30:52,086] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 12]: expected value 6, actual: 6 [2018-01-25 21:30:52,086] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 12]: expected value 6, actual: 6 [2018-01-25 21:30:52,086] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 13]: expected value 6, actual: 6 [2018-01-25 21:30:52,086] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 13]: expected value 6, actual: 6 [2018-01-25 21:30:52,087] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 14]: expected value 6, actual: 6 [2018-01-25 21:30:52,087] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 14]: expected value 6, actual: 6 [2018-01-25 21:30:52,087] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 15]: expected value 6, actual: 6 [2018-01-25 21:30:52,087] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 15]: expected value 6, actual: 6 [2018-01-25 21:30:52,087] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 16]: expected value 6, actual: 6 [2018-01-25 21:30:52,087] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 16]: expected value 6, actual: 6 [2018-01-25 21:30:52,088] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 17]: expected value 6, actual: 6 [2018-01-25 21:30:52,088] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 17]: expected value 6, actual: 6 [2018-01-25 21:30:52,088] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 18]: expected value 6, actual: 6 [2018-01-25 21:30:52,088] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 18]: expected value 6, actual: 6 [2018-01-25 21:30:52,088] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 19]: expected value 6, actual: 6 [2018-01-25 21:30:52,088] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 19]: expected value 6, actual: 6 [2018-01-25 21:30:52,089] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 20]: expected value 6, actual: 6 [2018-01-25 21:30:52,089] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 20]: expected value 6, actual: 6 [2018-01-25 21:30:52,089] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 21]: expected value 6, actual: 6 [2018-01-25 21:30:52,090] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 21]: expected value 6, actual: 6 [2018-01-25 21:30:52,090] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 22]: expected value 6, actual: 6 [2018-01-25 21:30:52,090] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 22]: expected value 6, actual: 6 [2018-01-25 21:30:52,090] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 23]: expected value 6, actual: 6 [2018-01-25 21:30:52,091] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 23]: expected value 6, actual: 6 [2018-01-25 21:30:52,091] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 24]: expected value 6, actual: 6 [2018-01-25 21:30:52,091] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 24]: expected value 6, actual: 6 [2018-01-25 21:30:52,091] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 25]: expected value 6, actual: 6 [2018-01-25 21:30:52,092] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 25]: expected value 6, actual: 6 [2018-01-25 21:30:52,092] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 26]: expected value 6, actual: 6 [2018-01-25 21:30:52,092] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 26]: expected value 6, actual: 6 [2018-01-25 21:30:52,092] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 27]: expected value 6, actual: 6 [2018-01-25 21:30:52,092] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 27]: expected value 6, actual: 6 [2018-01-25 21:30:52,093] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 28]: expected value 6, actual: 6 [2018-01-25 21:30:52,093] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group': 'true'}, server=172.17.0.8 Key [2008, 12, 28]: expected value 6, actual: 6 [2018-01-25 21:30:52,640] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:52,641] - [viewquerytests:2159] INFO - DDoc=test_view-e8b4520; View={"test_view-e8b4520": {"map": "function (doc) { if(doc.job_title !== undefined) emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] ); }", "reduce": "_count"}};:{'group_level': '1'}: Generated results contains 1 items [2018-01-25 21:30:53,642] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-e8b4520/_view/test_view-e8b4520?connection_timeout=60000&group_level=1 [2018-01-25 21:30:53,656] - [task:2724] INFO - view test_view-e8b4520, query {'connection_timeout': 60000, 'group_level': '1'}: expected- 1, actual -1 [2018-01-25 21:30:53,656] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '1'}, server=172.17.0.8 Key [2008]: expected value 2016, actual: 2016 [2018-01-25 21:30:53,657] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '1'}, server=172.17.0.8 Key [2008]: expected value 2016, actual: 2016 [2018-01-25 21:30:53,657] - [viewquerytests:2143] INFO - DDoc=test_view-e8b4520; View={"test_view-e8b4520": {"map": "function (doc) { if(doc.job_title !== undefined) emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] ); }", "reduce": "_count"}};: query={'group_level': '2'}: Generate results [2018-01-25 21:30:54,884] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:54,884] - [viewquerytests:2159] INFO - DDoc=test_view-e8b4520; View={"test_view-e8b4520": {"map": "function (doc) { if(doc.job_title !== undefined) emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] ); }", "reduce": "_count"}};:{'group_level': '2'}: Generated results contains 12 items [2018-01-25 21:30:55,885] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-e8b4520/_view/test_view-e8b4520?connection_timeout=60000&group_level=2 [2018-01-25 21:30:55,907] - [task:2724] INFO - view test_view-e8b4520, query {'connection_timeout': 60000, 'group_level': '2'}: expected- 12, actual -12 [2018-01-25 21:30:55,907] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '2'}, server=172.17.0.8 Key [2008, 1]: expected value 168, actual: 168 [2018-01-25 21:30:55,908] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '2'}, server=172.17.0.8 Key [2008, 1]: expected value 168, actual: 168 [2018-01-25 21:30:55,909] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '2'}, server=172.17.0.8 Key [2008, 2]: expected value 168, actual: 168 [2018-01-25 21:30:55,909] - [viewquerytests:2143] INFO - DDoc=test_view-e8b4520; View={"test_view-e8b4520": {"map": "function (doc) { if(doc.job_title !== undefined) emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] ); }", "reduce": "_count"}};: query={'group_level': '3'}: Generate results [2018-01-25 21:30:55,910] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '2'}, server=172.17.0.8 Key [2008, 2]: expected value 168, actual: 168 [2018-01-25 21:30:55,911] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '2'}, server=172.17.0.8 Key [2008, 3]: expected value 168, actual: 168 [2018-01-25 21:30:55,911] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '2'}, server=172.17.0.8 Key [2008, 3]: expected value 168, actual: 168 [2018-01-25 21:30:55,911] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '2'}, server=172.17.0.8 Key [2008, 4]: expected value 168, actual: 168 [2018-01-25 21:30:55,911] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '2'}, server=172.17.0.8 Key [2008, 4]: expected value 168, actual: 168 [2018-01-25 21:30:55,911] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '2'}, server=172.17.0.8 Key [2008, 5]: expected value 168, actual: 168 [2018-01-25 21:30:55,912] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '2'}, server=172.17.0.8 Key [2008, 5]: expected value 168, actual: 168 [2018-01-25 21:30:55,912] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '2'}, server=172.17.0.8 Key [2008, 6]: expected value 168, actual: 168 [2018-01-25 21:30:55,912] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '2'}, server=172.17.0.8 Key [2008, 6]: expected value 168, actual: 168 [2018-01-25 21:30:55,912] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '2'}, server=172.17.0.8 Key [2008, 7]: expected value 168, actual: 168 [2018-01-25 21:30:55,912] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '2'}, server=172.17.0.8 Key [2008, 7]: expected value 168, actual: 168 [2018-01-25 21:30:55,913] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '2'}, server=172.17.0.8 Key [2008, 8]: expected value 168, actual: 168 [2018-01-25 21:30:55,913] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '2'}, server=172.17.0.8 Key [2008, 8]: expected value 168, actual: 168 [2018-01-25 21:30:55,913] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '2'}, server=172.17.0.8 Key [2008, 9]: expected value 168, actual: 168 [2018-01-25 21:30:55,913] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '2'}, server=172.17.0.8 Key [2008, 9]: expected value 168, actual: 168 [2018-01-25 21:30:55,913] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '2'}, server=172.17.0.8 Key [2008, 10]: expected value 168, actual: 168 [2018-01-25 21:30:55,913] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '2'}, server=172.17.0.8 Key [2008, 10]: expected value 168, actual: 168 [2018-01-25 21:30:55,914] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '2'}, server=172.17.0.8 Key [2008, 11]: expected value 168, actual: 168 [2018-01-25 21:30:55,914] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '2'}, server=172.17.0.8 Key [2008, 11]: expected value 168, actual: 168 [2018-01-25 21:30:55,914] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '2'}, server=172.17.0.8 Key [2008, 12]: expected value 168, actual: 168 [2018-01-25 21:30:55,915] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '2'}, server=172.17.0.8 Key [2008, 12]: expected value 168, actual: 168 [2018-01-25 21:30:56,080] - [task:3397] INFO - Finished generating expected query results [2018-01-25 21:30:56,081] - [viewquerytests:2159] INFO - DDoc=test_view-e8b4520; View={"test_view-e8b4520": {"map": "function (doc) { if(doc.job_title !== undefined) emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] ); }", "reduce": "_count"}};:{'group_level': '3'}: Generated results contains 336 items [2018-01-25 21:30:57,083] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/test_view-e8b4520/_view/test_view-e8b4520?connection_timeout=60000&group_level=3 [2018-01-25 21:30:57,127] - [task:2724] INFO - view test_view-e8b4520, query {'connection_timeout': 60000, 'group_level': '3'}: expected- 336, actual -336 [2018-01-25 21:30:57,127] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 1]: expected value 6, actual: 6 [2018-01-25 21:30:57,127] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 1]: expected value 6, actual: 6 [2018-01-25 21:30:57,128] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 2]: expected value 6, actual: 6 [2018-01-25 21:30:57,128] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 2]: expected value 6, actual: 6 [2018-01-25 21:30:57,131] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 3]: expected value 6, actual: 6 [2018-01-25 21:30:57,131] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 3]: expected value 6, actual: 6 [2018-01-25 21:30:57,131] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 4]: expected value 6, actual: 6 [2018-01-25 21:30:57,131] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 4]: expected value 6, actual: 6 [2018-01-25 21:30:57,133] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 5]: expected value 6, actual: 6 [2018-01-25 21:30:57,133] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 5]: expected value 6, actual: 6 [2018-01-25 21:30:57,133] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 6]: expected value 6, actual: 6 [2018-01-25 21:30:57,134] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 6]: expected value 6, actual: 6 [2018-01-25 21:30:57,134] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 7]: expected value 6, actual: 6 [2018-01-25 21:30:57,135] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 7]: expected value 6, actual: 6 [2018-01-25 21:30:57,136] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 8]: expected value 6, actual: 6 [2018-01-25 21:30:57,136] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 8]: expected value 6, actual: 6 [2018-01-25 21:30:57,137] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 9]: expected value 6, actual: 6 [2018-01-25 21:30:57,138] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 9]: expected value 6, actual: 6 [2018-01-25 21:30:57,139] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 10]: expected value 6, actual: 6 [2018-01-25 21:30:57,139] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 10]: expected value 6, actual: 6 [2018-01-25 21:30:57,139] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 11]: expected value 6, actual: 6 [2018-01-25 21:30:57,139] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 11]: expected value 6, actual: 6 [2018-01-25 21:30:57,140] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 12]: expected value 6, actual: 6 [2018-01-25 21:30:57,140] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 12]: expected value 6, actual: 6 [2018-01-25 21:30:57,141] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 13]: expected value 6, actual: 6 [2018-01-25 21:30:57,141] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 13]: expected value 6, actual: 6 [2018-01-25 21:30:57,141] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 14]: expected value 6, actual: 6 [2018-01-25 21:30:57,142] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 14]: expected value 6, actual: 6 [2018-01-25 21:30:57,142] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 15]: expected value 6, actual: 6 [2018-01-25 21:30:57,143] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 15]: expected value 6, actual: 6 [2018-01-25 21:30:57,143] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 16]: expected value 6, actual: 6 [2018-01-25 21:30:57,143] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 16]: expected value 6, actual: 6 [2018-01-25 21:30:57,144] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 17]: expected value 6, actual: 6 [2018-01-25 21:30:57,144] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 17]: expected value 6, actual: 6 [2018-01-25 21:30:57,145] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 18]: expected value 6, actual: 6 [2018-01-25 21:30:57,146] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 18]: expected value 6, actual: 6 [2018-01-25 21:30:57,147] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 19]: expected value 6, actual: 6 [2018-01-25 21:30:57,147] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 19]: expected value 6, actual: 6 [2018-01-25 21:30:57,147] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 20]: expected value 6, actual: 6 [2018-01-25 21:30:57,148] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 20]: expected value 6, actual: 6 [2018-01-25 21:30:57,148] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 21]: expected value 6, actual: 6 [2018-01-25 21:30:57,149] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 21]: expected value 6, actual: 6 [2018-01-25 21:30:57,149] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 22]: expected value 6, actual: 6 [2018-01-25 21:30:57,150] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 22]: expected value 6, actual: 6 [2018-01-25 21:30:57,151] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 23]: expected value 6, actual: 6 [2018-01-25 21:30:57,151] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 23]: expected value 6, actual: 6 [2018-01-25 21:30:57,153] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 24]: expected value 6, actual: 6 [2018-01-25 21:30:57,153] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 24]: expected value 6, actual: 6 [2018-01-25 21:30:57,154] - [viewquerytests:2047] INFO - Current amount of threads 0 [2018-01-25 21:30:57,154] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 25]: expected value 6, actual: 6 [2018-01-25 21:30:57,163] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 25]: expected value 6, actual: 6 [2018-01-25 21:30:57,163] - [basetestcase:414] INFO - ============== basetestcase cleanup was started for test #14 test_employee_dataset_all_queries ============== [2018-01-25 21:30:57,165] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 26]: expected value 6, actual: 6 [2018-01-25 21:30:57,167] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 26]: expected value 6, actual: 6 [2018-01-25 21:30:57,168] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 27]: expected value 6, actual: 6 [2018-01-25 21:30:57,168] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 27]: expected value 6, actual: 6 [2018-01-25 21:30:57,169] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 28]: expected value 6, actual: 6 [2018-01-25 21:30:57,170] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 1, 28]: expected value 6, actual: 6 [2018-01-25 21:30:57,170] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 1]: expected value 6, actual: 6 [2018-01-25 21:30:57,170] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 1]: expected value 6, actual: 6 [2018-01-25 21:30:57,171] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 2]: expected value 6, actual: 6 [2018-01-25 21:30:57,171] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 2]: expected value 6, actual: 6 [2018-01-25 21:30:57,171] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 3]: expected value 6, actual: 6 [2018-01-25 21:30:57,171] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 3]: expected value 6, actual: 6 [2018-01-25 21:30:57,173] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 4]: expected value 6, actual: 6 [2018-01-25 21:30:57,173] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 4]: expected value 6, actual: 6 [2018-01-25 21:30:57,173] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 5]: expected value 6, actual: 6 [2018-01-25 21:30:57,174] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 5]: expected value 6, actual: 6 [2018-01-25 21:30:57,174] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 6]: expected value 6, actual: 6 [2018-01-25 21:30:57,174] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 6]: expected value 6, actual: 6 [2018-01-25 21:30:57,174] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 7]: expected value 6, actual: 6 [2018-01-25 21:30:57,175] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 7]: expected value 6, actual: 6 [2018-01-25 21:30:57,175] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 8]: expected value 6, actual: 6 [2018-01-25 21:30:57,175] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 8]: expected value 6, actual: 6 [2018-01-25 21:30:57,185] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 9]: expected value 6, actual: 6 [2018-01-25 21:30:57,185] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 9]: expected value 6, actual: 6 [2018-01-25 21:30:57,186] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 10]: expected value 6, actual: 6 [2018-01-25 21:30:57,186] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 10]: expected value 6, actual: 6 [2018-01-25 21:30:57,187] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 11]: expected value 6, actual: 6 [2018-01-25 21:30:57,187] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 11]: expected value 6, actual: 6 [2018-01-25 21:30:57,188] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 12]: expected value 6, actual: 6 [2018-01-25 21:30:57,188] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 12]: expected value 6, actual: 6 [2018-01-25 21:30:57,188] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 13]: expected value 6, actual: 6 [2018-01-25 21:30:57,189] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 13]: expected value 6, actual: 6 [2018-01-25 21:30:57,189] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 14]: expected value 6, actual: 6 [2018-01-25 21:30:57,189] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 14]: expected value 6, actual: 6 [2018-01-25 21:30:57,190] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 15]: expected value 6, actual: 6 [2018-01-25 21:30:57,190] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 15]: expected value 6, actual: 6 [2018-01-25 21:30:57,191] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 16]: expected value 6, actual: 6 [2018-01-25 21:30:57,192] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 16]: expected value 6, actual: 6 [2018-01-25 21:30:57,194] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 17]: expected value 6, actual: 6 [2018-01-25 21:30:57,194] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 17]: expected value 6, actual: 6 [2018-01-25 21:30:57,195] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 18]: expected value 6, actual: 6 [2018-01-25 21:30:57,195] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 18]: expected value 6, actual: 6 [2018-01-25 21:30:57,196] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 19]: expected value 6, actual: 6 [2018-01-25 21:30:57,196] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 19]: expected value 6, actual: 6 [2018-01-25 21:30:57,197] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 20]: expected value 6, actual: 6 [2018-01-25 21:30:57,197] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 20]: expected value 6, actual: 6 [2018-01-25 21:30:57,198] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 21]: expected value 6, actual: 6 [2018-01-25 21:30:57,198] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 21]: expected value 6, actual: 6 [2018-01-25 21:30:57,199] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 22]: expected value 6, actual: 6 [2018-01-25 21:30:57,199] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 22]: expected value 6, actual: 6 [2018-01-25 21:30:57,199] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 23]: expected value 6, actual: 6 [2018-01-25 21:30:57,200] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 23]: expected value 6, actual: 6 [2018-01-25 21:30:57,201] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 24]: expected value 6, actual: 6 [2018-01-25 21:30:57,201] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 24]: expected value 6, actual: 6 [2018-01-25 21:30:57,202] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 25]: expected value 6, actual: 6 [2018-01-25 21:30:57,203] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 25]: expected value 6, actual: 6 [2018-01-25 21:30:57,204] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 26]: expected value 6, actual: 6 [2018-01-25 21:30:57,204] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 26]: expected value 6, actual: 6 [2018-01-25 21:30:57,205] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 27]: expected value 6, actual: 6 [2018-01-25 21:30:57,205] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 27]: expected value 6, actual: 6 [2018-01-25 21:30:57,206] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 28]: expected value 6, actual: 6 [2018-01-25 21:30:57,206] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 2, 28]: expected value 6, actual: 6 [2018-01-25 21:30:57,207] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 1]: expected value 6, actual: 6 [2018-01-25 21:30:57,207] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 1]: expected value 6, actual: 6 [2018-01-25 21:30:57,207] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 2]: expected value 6, actual: 6 [2018-01-25 21:30:57,208] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 2]: expected value 6, actual: 6 [2018-01-25 21:30:57,209] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 3]: expected value 6, actual: 6 [2018-01-25 21:30:57,210] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 3]: expected value 6, actual: 6 [2018-01-25 21:30:57,211] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 4]: expected value 6, actual: 6 [2018-01-25 21:30:57,211] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 4]: expected value 6, actual: 6 [2018-01-25 21:30:57,213] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 5]: expected value 6, actual: 6 [2018-01-25 21:30:57,213] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 5]: expected value 6, actual: 6 [2018-01-25 21:30:57,214] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 6]: expected value 6, actual: 6 [2018-01-25 21:30:57,214] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 6]: expected value 6, actual: 6 [2018-01-25 21:30:57,214] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 7]: expected value 6, actual: 6 [2018-01-25 21:30:57,215] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 7]: expected value 6, actual: 6 [2018-01-25 21:30:57,215] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 8]: expected value 6, actual: 6 [2018-01-25 21:30:57,216] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 8]: expected value 6, actual: 6 [2018-01-25 21:30:57,217] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 9]: expected value 6, actual: 6 [2018-01-25 21:30:57,217] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 9]: expected value 6, actual: 6 [2018-01-25 21:30:57,219] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 10]: expected value 6, actual: 6 [2018-01-25 21:30:57,219] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 10]: expected value 6, actual: 6 [2018-01-25 21:30:57,220] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 11]: expected value 6, actual: 6 [2018-01-25 21:30:57,220] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 11]: expected value 6, actual: 6 [2018-01-25 21:30:57,222] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 12]: expected value 6, actual: 6 [2018-01-25 21:30:57,222] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 12]: expected value 6, actual: 6 [2018-01-25 21:30:57,222] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 13]: expected value 6, actual: 6 [2018-01-25 21:30:57,222] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 13]: expected value 6, actual: 6 [2018-01-25 21:30:57,223] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 14]: expected value 6, actual: 6 [2018-01-25 21:30:57,223] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 14]: expected value 6, actual: 6 [2018-01-25 21:30:57,223] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 15]: expected value 6, actual: 6 [2018-01-25 21:30:57,224] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 15]: expected value 6, actual: 6 [2018-01-25 21:30:57,224] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 16]: expected value 6, actual: 6 [2018-01-25 21:30:57,224] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 16]: expected value 6, actual: 6 [2018-01-25 21:30:57,225] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 17]: expected value 6, actual: 6 [2018-01-25 21:30:57,225] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 17]: expected value 6, actual: 6 [2018-01-25 21:30:57,226] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 18]: expected value 6, actual: 6 [2018-01-25 21:30:57,226] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 18]: expected value 6, actual: 6 [2018-01-25 21:30:57,227] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 19]: expected value 6, actual: 6 [2018-01-25 21:30:57,227] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 19]: expected value 6, actual: 6 [2018-01-25 21:30:57,227] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 20]: expected value 6, actual: 6 [2018-01-25 21:30:57,228] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 20]: expected value 6, actual: 6 [2018-01-25 21:30:57,228] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 21]: expected value 6, actual: 6 [2018-01-25 21:30:57,228] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 21]: expected value 6, actual: 6 [2018-01-25 21:30:57,229] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 22]: expected value 6, actual: 6 [2018-01-25 21:30:57,229] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 22]: expected value 6, actual: 6 [2018-01-25 21:30:57,229] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 23]: expected value 6, actual: 6 [2018-01-25 21:30:57,230] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 23]: expected value 6, actual: 6 [2018-01-25 21:30:57,230] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 24]: expected value 6, actual: 6 [2018-01-25 21:30:57,230] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 24]: expected value 6, actual: 6 [2018-01-25 21:30:57,230] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 25]: expected value 6, actual: 6 [2018-01-25 21:30:57,231] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 25]: expected value 6, actual: 6 [2018-01-25 21:30:57,232] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 26]: expected value 6, actual: 6 [2018-01-25 21:30:57,233] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 26]: expected value 6, actual: 6 [2018-01-25 21:30:57,233] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 27]: expected value 6, actual: 6 [2018-01-25 21:30:57,234] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 27]: expected value 6, actual: 6 [2018-01-25 21:30:57,234] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 28]: expected value 6, actual: 6 [2018-01-25 21:30:57,235] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 3, 28]: expected value 6, actual: 6 [2018-01-25 21:30:57,235] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 1]: expected value 6, actual: 6 [2018-01-25 21:30:57,333] - [basetestcase:422] WARNING - Alerts were found: [{u'msg': u"IP address seems to have changed. Unable to listen on 'ns_1@cb-example-0003.cb-example.default.svc'. (POSIX error code: 'nxdomain')", u'serverTime': u'2018-01-25T21:28:43.000Z'}] [2018-01-25 21:30:57,277] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 1]: expected value 6, actual: 6 [2018-01-25 21:30:57,357] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 2]: expected value 6, actual: 6 [2018-01-25 21:30:57,358] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 2]: expected value 6, actual: 6 [2018-01-25 21:30:57,359] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 3]: expected value 6, actual: 6 [2018-01-25 21:30:57,359] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 3]: expected value 6, actual: 6 [2018-01-25 21:30:57,360] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 4]: expected value 6, actual: 6 [2018-01-25 21:30:57,361] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 4]: expected value 6, actual: 6 [2018-01-25 21:30:57,361] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 5]: expected value 6, actual: 6 [2018-01-25 21:30:57,362] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 5]: expected value 6, actual: 6 [2018-01-25 21:30:57,363] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 6]: expected value 6, actual: 6 [2018-01-25 21:30:57,363] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 6]: expected value 6, actual: 6 [2018-01-25 21:30:57,364] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 7]: expected value 6, actual: 6 [2018-01-25 21:30:57,364] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 7]: expected value 6, actual: 6 [2018-01-25 21:30:57,364] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 8]: expected value 6, actual: 6 [2018-01-25 21:30:57,365] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 8]: expected value 6, actual: 6 [2018-01-25 21:30:57,365] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 9]: expected value 6, actual: 6 [2018-01-25 21:30:57,366] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 9]: expected value 6, actual: 6 [2018-01-25 21:30:57,367] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 10]: expected value 6, actual: 6 [2018-01-25 21:30:57,368] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 10]: expected value 6, actual: 6 [2018-01-25 21:30:57,368] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 11]: expected value 6, actual: 6 [2018-01-25 21:30:57,368] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 11]: expected value 6, actual: 6 [2018-01-25 21:30:57,369] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 12]: expected value 6, actual: 6 [2018-01-25 21:30:57,369] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 12]: expected value 6, actual: 6 [2018-01-25 21:30:57,370] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 13]: expected value 6, actual: 6 [2018-01-25 21:30:57,370] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 13]: expected value 6, actual: 6 [2018-01-25 21:30:57,370] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 14]: expected value 6, actual: 6 [2018-01-25 21:30:57,371] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 14]: expected value 6, actual: 6 [2018-01-25 21:30:57,371] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 15]: expected value 6, actual: 6 [2018-01-25 21:30:57,371] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 15]: expected value 6, actual: 6 [2018-01-25 21:30:57,372] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 16]: expected value 6, actual: 6 [2018-01-25 21:30:57,372] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 16]: expected value 6, actual: 6 [2018-01-25 21:30:57,373] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 17]: expected value 6, actual: 6 [2018-01-25 21:30:57,373] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 17]: expected value 6, actual: 6 [2018-01-25 21:30:57,374] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 18]: expected value 6, actual: 6 [2018-01-25 21:30:57,374] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 18]: expected value 6, actual: 6 [2018-01-25 21:30:57,375] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 19]: expected value 6, actual: 6 [2018-01-25 21:30:57,375] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 19]: expected value 6, actual: 6 [2018-01-25 21:30:57,375] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 20]: expected value 6, actual: 6 [2018-01-25 21:30:57,375] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 20]: expected value 6, actual: 6 [2018-01-25 21:30:57,376] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 21]: expected value 6, actual: 6 [2018-01-25 21:30:57,376] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 21]: expected value 6, actual: 6 [2018-01-25 21:30:57,376] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 22]: expected value 6, actual: 6 [2018-01-25 21:30:57,377] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 22]: expected value 6, actual: 6 [2018-01-25 21:30:57,377] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 23]: expected value 6, actual: 6 [2018-01-25 21:30:57,377] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 23]: expected value 6, actual: 6 [2018-01-25 21:30:57,377] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 24]: expected value 6, actual: 6 [2018-01-25 21:30:57,377] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 24]: expected value 6, actual: 6 [2018-01-25 21:30:57,378] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 25]: expected value 6, actual: 6 [2018-01-25 21:30:57,378] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 25]: expected value 6, actual: 6 [2018-01-25 21:30:57,378] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 26]: expected value 6, actual: 6 [2018-01-25 21:30:57,378] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 26]: expected value 6, actual: 6 [2018-01-25 21:30:57,378] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 27]: expected value 6, actual: 6 [2018-01-25 21:30:57,378] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 27]: expected value 6, actual: 6 [2018-01-25 21:30:57,378] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 28]: expected value 6, actual: 6 [2018-01-25 21:30:57,379] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 4, 28]: expected value 6, actual: 6 [2018-01-25 21:30:57,379] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 1]: expected value 6, actual: 6 [2018-01-25 21:30:57,379] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 1]: expected value 6, actual: 6 [2018-01-25 21:30:57,379] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 2]: expected value 6, actual: 6 [2018-01-25 21:30:57,379] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 2]: expected value 6, actual: 6 [2018-01-25 21:30:57,380] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 3]: expected value 6, actual: 6 [2018-01-25 21:30:57,381] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 3]: expected value 6, actual: 6 [2018-01-25 21:30:57,381] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 4]: expected value 6, actual: 6 [2018-01-25 21:30:57,381] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 4]: expected value 6, actual: 6 [2018-01-25 21:30:57,381] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 5]: expected value 6, actual: 6 [2018-01-25 21:30:57,382] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 5]: expected value 6, actual: 6 [2018-01-25 21:30:57,382] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 6]: expected value 6, actual: 6 [2018-01-25 21:30:57,382] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 6]: expected value 6, actual: 6 [2018-01-25 21:30:57,383] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 7]: expected value 6, actual: 6 [2018-01-25 21:30:57,383] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 7]: expected value 6, actual: 6 [2018-01-25 21:30:57,383] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 8]: expected value 6, actual: 6 [2018-01-25 21:30:57,383] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 8]: expected value 6, actual: 6 [2018-01-25 21:30:57,383] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 9]: expected value 6, actual: 6 [2018-01-25 21:30:57,383] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 9]: expected value 6, actual: 6 [2018-01-25 21:30:57,384] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 10]: expected value 6, actual: 6 [2018-01-25 21:30:57,384] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 10]: expected value 6, actual: 6 [2018-01-25 21:30:57,384] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 11]: expected value 6, actual: 6 [2018-01-25 21:30:57,384] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 11]: expected value 6, actual: 6 [2018-01-25 21:30:57,385] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 12]: expected value 6, actual: 6 [2018-01-25 21:30:57,385] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 12]: expected value 6, actual: 6 [2018-01-25 21:30:57,385] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 13]: expected value 6, actual: 6 [2018-01-25 21:30:57,385] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 13]: expected value 6, actual: 6 [2018-01-25 21:30:57,386] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 14]: expected value 6, actual: 6 [2018-01-25 21:30:57,386] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 14]: expected value 6, actual: 6 [2018-01-25 21:30:57,386] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 15]: expected value 6, actual: 6 [2018-01-25 21:30:57,386] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 15]: expected value 6, actual: 6 [2018-01-25 21:30:57,387] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 16]: expected value 6, actual: 6 [2018-01-25 21:30:57,387] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 16]: expected value 6, actual: 6 [2018-01-25 21:30:57,388] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 17]: expected value 6, actual: 6 [2018-01-25 21:30:57,389] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 17]: expected value 6, actual: 6 [2018-01-25 21:30:57,389] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 18]: expected value 6, actual: 6 [2018-01-25 21:30:57,389] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 18]: expected value 6, actual: 6 [2018-01-25 21:30:57,390] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 19]: expected value 6, actual: 6 [2018-01-25 21:30:57,390] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 19]: expected value 6, actual: 6 [2018-01-25 21:30:57,390] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 20]: expected value 6, actual: 6 [2018-01-25 21:30:57,390] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 20]: expected value 6, actual: 6 [2018-01-25 21:30:57,391] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 21]: expected value 6, actual: 6 [2018-01-25 21:30:57,391] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 21]: expected value 6, actual: 6 [2018-01-25 21:30:57,391] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 22]: expected value 6, actual: 6 [2018-01-25 21:30:57,391] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 22]: expected value 6, actual: 6 [2018-01-25 21:30:57,391] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 23]: expected value 6, actual: 6 [2018-01-25 21:30:57,392] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 23]: expected value 6, actual: 6 [2018-01-25 21:30:57,392] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 24]: expected value 6, actual: 6 [2018-01-25 21:30:57,392] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 24]: expected value 6, actual: 6 [2018-01-25 21:30:57,392] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 25]: expected value 6, actual: 6 [2018-01-25 21:30:57,392] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 25]: expected value 6, actual: 6 [2018-01-25 21:30:57,392] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 26]: expected value 6, actual: 6 [2018-01-25 21:30:57,392] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 26]: expected value 6, actual: 6 [2018-01-25 21:30:57,393] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 27]: expected value 6, actual: 6 [2018-01-25 21:30:57,393] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 27]: expected value 6, actual: 6 [2018-01-25 21:30:57,393] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 28]: expected value 6, actual: 6 [2018-01-25 21:30:57,393] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 5, 28]: expected value 6, actual: 6 [2018-01-25 21:30:57,393] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 1]: expected value 6, actual: 6 [2018-01-25 21:30:57,393] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 1]: expected value 6, actual: 6 [2018-01-25 21:30:57,393] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 2]: expected value 6, actual: 6 [2018-01-25 21:30:57,393] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 2]: expected value 6, actual: 6 [2018-01-25 21:30:57,394] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 3]: expected value 6, actual: 6 [2018-01-25 21:30:57,394] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 3]: expected value 6, actual: 6 [2018-01-25 21:30:57,394] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 4]: expected value 6, actual: 6 [2018-01-25 21:30:57,394] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 4]: expected value 6, actual: 6 [2018-01-25 21:30:57,394] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 5]: expected value 6, actual: 6 [2018-01-25 21:30:57,394] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 5]: expected value 6, actual: 6 [2018-01-25 21:30:57,394] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 6]: expected value 6, actual: 6 [2018-01-25 21:30:57,394] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 6]: expected value 6, actual: 6 [2018-01-25 21:30:57,395] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 7]: expected value 6, actual: 6 [2018-01-25 21:30:57,395] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 7]: expected value 6, actual: 6 [2018-01-25 21:30:57,395] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 8]: expected value 6, actual: 6 [2018-01-25 21:30:57,395] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 8]: expected value 6, actual: 6 [2018-01-25 21:30:57,395] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 9]: expected value 6, actual: 6 [2018-01-25 21:30:57,395] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 9]: expected value 6, actual: 6 [2018-01-25 21:30:57,395] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 10]: expected value 6, actual: 6 [2018-01-25 21:30:57,396] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 10]: expected value 6, actual: 6 [2018-01-25 21:30:57,396] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 11]: expected value 6, actual: 6 [2018-01-25 21:30:57,396] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 11]: expected value 6, actual: 6 [2018-01-25 21:30:57,396] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 12]: expected value 6, actual: 6 [2018-01-25 21:30:57,396] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 12]: expected value 6, actual: 6 [2018-01-25 21:30:57,396] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 13]: expected value 6, actual: 6 [2018-01-25 21:30:57,396] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 13]: expected value 6, actual: 6 [2018-01-25 21:30:57,396] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 14]: expected value 6, actual: 6 [2018-01-25 21:30:57,397] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 14]: expected value 6, actual: 6 [2018-01-25 21:30:57,397] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 15]: expected value 6, actual: 6 [2018-01-25 21:30:57,397] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 15]: expected value 6, actual: 6 [2018-01-25 21:30:57,397] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 16]: expected value 6, actual: 6 [2018-01-25 21:30:57,397] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 16]: expected value 6, actual: 6 [2018-01-25 21:30:57,397] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 17]: expected value 6, actual: 6 [2018-01-25 21:30:57,397] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 17]: expected value 6, actual: 6 [2018-01-25 21:30:57,398] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 18]: expected value 6, actual: 6 [2018-01-25 21:30:57,398] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 18]: expected value 6, actual: 6 [2018-01-25 21:30:57,398] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 19]: expected value 6, actual: 6 [2018-01-25 21:30:57,398] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 19]: expected value 6, actual: 6 [2018-01-25 21:30:57,398] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 20]: expected value 6, actual: 6 [2018-01-25 21:30:57,398] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 20]: expected value 6, actual: 6 [2018-01-25 21:30:57,398] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 21]: expected value 6, actual: 6 [2018-01-25 21:30:57,398] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 21]: expected value 6, actual: 6 [2018-01-25 21:30:57,399] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 22]: expected value 6, actual: 6 [2018-01-25 21:30:57,399] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 22]: expected value 6, actual: 6 [2018-01-25 21:30:57,399] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 23]: expected value 6, actual: 6 [2018-01-25 21:30:57,399] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 23]: expected value 6, actual: 6 [2018-01-25 21:30:57,399] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 24]: expected value 6, actual: 6 [2018-01-25 21:30:57,399] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 24]: expected value 6, actual: 6 [2018-01-25 21:30:57,399] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 25]: expected value 6, actual: 6 [2018-01-25 21:30:57,400] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 25]: expected value 6, actual: 6 [2018-01-25 21:30:57,400] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 26]: expected value 6, actual: 6 [2018-01-25 21:30:57,400] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 26]: expected value 6, actual: 6 [2018-01-25 21:30:57,401] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 27]: expected value 6, actual: 6 [2018-01-25 21:30:57,401] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 27]: expected value 6, actual: 6 [2018-01-25 21:30:57,402] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 28]: expected value 6, actual: 6 [2018-01-25 21:30:57,402] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 6, 28]: expected value 6, actual: 6 [2018-01-25 21:30:57,402] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 1]: expected value 6, actual: 6 [2018-01-25 21:30:57,402] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 1]: expected value 6, actual: 6 [2018-01-25 21:30:57,403] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 2]: expected value 6, actual: 6 [2018-01-25 21:30:57,403] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 2]: expected value 6, actual: 6 [2018-01-25 21:30:57,403] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 3]: expected value 6, actual: 6 [2018-01-25 21:30:57,403] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 3]: expected value 6, actual: 6 [2018-01-25 21:30:57,404] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 4]: expected value 6, actual: 6 [2018-01-25 21:30:57,404] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 4]: expected value 6, actual: 6 [2018-01-25 21:30:57,404] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 5]: expected value 6, actual: 6 [2018-01-25 21:30:57,404] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 5]: expected value 6, actual: 6 [2018-01-25 21:30:57,404] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 6]: expected value 6, actual: 6 [2018-01-25 21:30:57,406] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 6]: expected value 6, actual: 6 [2018-01-25 21:30:57,406] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 7]: expected value 6, actual: 6 [2018-01-25 21:30:57,407] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 7]: expected value 6, actual: 6 [2018-01-25 21:30:57,409] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 8]: expected value 6, actual: 6 [2018-01-25 21:30:57,410] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 8]: expected value 6, actual: 6 [2018-01-25 21:30:57,411] - [bucket_helper:142] INFO - deleting existing buckets [u'default'] on 172.17.0.8 [2018-01-25 21:30:57,412] - [bucket_helper:144] INFO - remove bucket default ... [2018-01-25 21:30:57,413] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 9]: expected value 6, actual: 6 [2018-01-25 21:30:57,414] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 9]: expected value 6, actual: 6 [2018-01-25 21:30:57,416] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 10]: expected value 6, actual: 6 [2018-01-25 21:30:57,418] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 10]: expected value 6, actual: 6 [2018-01-25 21:30:57,420] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 11]: expected value 6, actual: 6 [2018-01-25 21:30:57,420] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 11]: expected value 6, actual: 6 [2018-01-25 21:30:57,422] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 12]: expected value 6, actual: 6 [2018-01-25 21:30:57,422] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 12]: expected value 6, actual: 6 [2018-01-25 21:30:57,422] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 13]: expected value 6, actual: 6 [2018-01-25 21:30:57,423] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 13]: expected value 6, actual: 6 [2018-01-25 21:30:57,423] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 14]: expected value 6, actual: 6 [2018-01-25 21:30:57,423] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 14]: expected value 6, actual: 6 [2018-01-25 21:30:57,424] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 15]: expected value 6, actual: 6 [2018-01-25 21:30:57,424] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 15]: expected value 6, actual: 6 [2018-01-25 21:30:57,425] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 16]: expected value 6, actual: 6 [2018-01-25 21:30:57,425] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 16]: expected value 6, actual: 6 [2018-01-25 21:30:57,426] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 17]: expected value 6, actual: 6 [2018-01-25 21:30:57,426] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 17]: expected value 6, actual: 6 [2018-01-25 21:30:57,427] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 18]: expected value 6, actual: 6 [2018-01-25 21:30:57,427] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 18]: expected value 6, actual: 6 [2018-01-25 21:30:57,428] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 19]: expected value 6, actual: 6 [2018-01-25 21:30:57,428] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 19]: expected value 6, actual: 6 [2018-01-25 21:30:57,430] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 20]: expected value 6, actual: 6 [2018-01-25 21:30:57,430] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 20]: expected value 6, actual: 6 [2018-01-25 21:30:57,431] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 21]: expected value 6, actual: 6 [2018-01-25 21:30:57,431] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 21]: expected value 6, actual: 6 [2018-01-25 21:30:57,431] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 22]: expected value 6, actual: 6 [2018-01-25 21:30:57,432] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 22]: expected value 6, actual: 6 [2018-01-25 21:30:57,432] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 23]: expected value 6, actual: 6 [2018-01-25 21:30:57,432] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 23]: expected value 6, actual: 6 [2018-01-25 21:30:57,434] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 24]: expected value 6, actual: 6 [2018-01-25 21:30:57,434] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 24]: expected value 6, actual: 6 [2018-01-25 21:30:57,434] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 25]: expected value 6, actual: 6 [2018-01-25 21:30:57,434] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 25]: expected value 6, actual: 6 [2018-01-25 21:30:57,435] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 26]: expected value 6, actual: 6 [2018-01-25 21:30:57,435] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 26]: expected value 6, actual: 6 [2018-01-25 21:30:57,435] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 27]: expected value 6, actual: 6 [2018-01-25 21:30:57,443] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 27]: expected value 6, actual: 6 [2018-01-25 21:30:57,460] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 28]: expected value 6, actual: 6 [2018-01-25 21:30:57,463] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 7, 28]: expected value 6, actual: 6 [2018-01-25 21:30:57,469] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 1]: expected value 6, actual: 6 [2018-01-25 21:30:57,478] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 1]: expected value 6, actual: 6 [2018-01-25 21:30:57,500] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 2]: expected value 6, actual: 6 [2018-01-25 21:30:57,500] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 2]: expected value 6, actual: 6 [2018-01-25 21:30:57,500] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 3]: expected value 6, actual: 6 [2018-01-25 21:30:57,509] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 3]: expected value 6, actual: 6 [2018-01-25 21:30:57,520] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 4]: expected value 6, actual: 6 [2018-01-25 21:30:57,543] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 4]: expected value 6, actual: 6 [2018-01-25 21:30:57,544] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 5]: expected value 6, actual: 6 [2018-01-25 21:30:57,544] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 5]: expected value 6, actual: 6 [2018-01-25 21:30:57,550] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 6]: expected value 6, actual: 6 [2018-01-25 21:30:57,550] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 6]: expected value 6, actual: 6 [2018-01-25 21:30:57,566] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 7]: expected value 6, actual: 6 [2018-01-25 21:30:57,567] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 7]: expected value 6, actual: 6 [2018-01-25 21:30:57,567] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 8]: expected value 6, actual: 6 [2018-01-25 21:30:57,567] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 8]: expected value 6, actual: 6 [2018-01-25 21:30:57,567] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 9]: expected value 6, actual: 6 [2018-01-25 21:30:57,567] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 9]: expected value 6, actual: 6 [2018-01-25 21:30:57,581] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 10]: expected value 6, actual: 6 [2018-01-25 21:30:57,581] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 10]: expected value 6, actual: 6 [2018-01-25 21:30:57,582] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 11]: expected value 6, actual: 6 [2018-01-25 21:30:57,582] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 11]: expected value 6, actual: 6 [2018-01-25 21:30:57,582] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 12]: expected value 6, actual: 6 [2018-01-25 21:30:57,587] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 12]: expected value 6, actual: 6 [2018-01-25 21:30:57,612] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 13]: expected value 6, actual: 6 [2018-01-25 21:30:57,620] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 13]: expected value 6, actual: 6 [2018-01-25 21:30:57,629] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 14]: expected value 6, actual: 6 [2018-01-25 21:30:57,629] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 14]: expected value 6, actual: 6 [2018-01-25 21:30:57,630] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 15]: expected value 6, actual: 6 [2018-01-25 21:30:57,630] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 15]: expected value 6, actual: 6 [2018-01-25 21:30:57,631] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 16]: expected value 6, actual: 6 [2018-01-25 21:30:57,632] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 16]: expected value 6, actual: 6 [2018-01-25 21:30:57,632] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 17]: expected value 6, actual: 6 [2018-01-25 21:30:57,632] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 17]: expected value 6, actual: 6 [2018-01-25 21:30:57,635] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 18]: expected value 6, actual: 6 [2018-01-25 21:30:57,642] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 18]: expected value 6, actual: 6 [2018-01-25 21:30:57,651] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 19]: expected value 6, actual: 6 [2018-01-25 21:30:57,651] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 19]: expected value 6, actual: 6 [2018-01-25 21:30:57,668] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 20]: expected value 6, actual: 6 [2018-01-25 21:30:57,680] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 20]: expected value 6, actual: 6 [2018-01-25 21:30:57,680] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 21]: expected value 6, actual: 6 [2018-01-25 21:30:57,680] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 21]: expected value 6, actual: 6 [2018-01-25 21:30:57,682] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 22]: expected value 6, actual: 6 [2018-01-25 21:30:57,682] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 22]: expected value 6, actual: 6 [2018-01-25 21:30:57,683] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 23]: expected value 6, actual: 6 [2018-01-25 21:30:57,683] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 23]: expected value 6, actual: 6 [2018-01-25 21:30:57,729] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 24]: expected value 6, actual: 6 [2018-01-25 21:30:57,730] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 24]: expected value 6, actual: 6 [2018-01-25 21:30:57,730] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 25]: expected value 6, actual: 6 [2018-01-25 21:30:57,730] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 25]: expected value 6, actual: 6 [2018-01-25 21:30:57,731] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 26]: expected value 6, actual: 6 [2018-01-25 21:30:57,736] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 26]: expected value 6, actual: 6 [2018-01-25 21:30:57,751] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 27]: expected value 6, actual: 6 [2018-01-25 21:30:57,777] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 27]: expected value 6, actual: 6 [2018-01-25 21:30:57,778] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 28]: expected value 6, actual: 6 [2018-01-25 21:30:57,778] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 8, 28]: expected value 6, actual: 6 [2018-01-25 21:30:57,822] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 1]: expected value 6, actual: 6 [2018-01-25 21:30:57,822] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 1]: expected value 6, actual: 6 [2018-01-25 21:30:57,824] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 2]: expected value 6, actual: 6 [2018-01-25 21:30:57,825] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 2]: expected value 6, actual: 6 [2018-01-25 21:30:57,826] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 3]: expected value 6, actual: 6 [2018-01-25 21:30:57,827] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 3]: expected value 6, actual: 6 [2018-01-25 21:30:57,828] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 4]: expected value 6, actual: 6 [2018-01-25 21:30:57,828] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 4]: expected value 6, actual: 6 [2018-01-25 21:30:57,829] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 5]: expected value 6, actual: 6 [2018-01-25 21:30:57,829] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 5]: expected value 6, actual: 6 [2018-01-25 21:30:57,829] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 6]: expected value 6, actual: 6 [2018-01-25 21:30:57,832] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 6]: expected value 6, actual: 6 [2018-01-25 21:30:57,833] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 7]: expected value 6, actual: 6 [2018-01-25 21:30:57,833] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 7]: expected value 6, actual: 6 [2018-01-25 21:30:57,833] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 8]: expected value 6, actual: 6 [2018-01-25 21:30:57,834] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 8]: expected value 6, actual: 6 [2018-01-25 21:30:57,834] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 9]: expected value 6, actual: 6 [2018-01-25 21:30:57,834] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 9]: expected value 6, actual: 6 [2018-01-25 21:30:57,835] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 10]: expected value 6, actual: 6 [2018-01-25 21:30:57,835] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 10]: expected value 6, actual: 6 [2018-01-25 21:30:57,835] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 11]: expected value 6, actual: 6 [2018-01-25 21:30:57,836] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 11]: expected value 6, actual: 6 [2018-01-25 21:30:57,836] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 12]: expected value 6, actual: 6 [2018-01-25 21:30:57,849] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 12]: expected value 6, actual: 6 [2018-01-25 21:30:57,850] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 13]: expected value 6, actual: 6 [2018-01-25 21:30:57,850] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 13]: expected value 6, actual: 6 [2018-01-25 21:30:57,850] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 14]: expected value 6, actual: 6 [2018-01-25 21:30:57,851] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 14]: expected value 6, actual: 6 [2018-01-25 21:30:57,851] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 15]: expected value 6, actual: 6 [2018-01-25 21:30:57,851] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 15]: expected value 6, actual: 6 [2018-01-25 21:30:57,870] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 16]: expected value 6, actual: 6 [2018-01-25 21:30:57,881] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 16]: expected value 6, actual: 6 [2018-01-25 21:30:57,881] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 17]: expected value 6, actual: 6 [2018-01-25 21:30:57,882] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 17]: expected value 6, actual: 6 [2018-01-25 21:30:57,882] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 18]: expected value 6, actual: 6 [2018-01-25 21:30:57,901] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 18]: expected value 6, actual: 6 [2018-01-25 21:30:57,904] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 19]: expected value 6, actual: 6 [2018-01-25 21:30:57,905] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 19]: expected value 6, actual: 6 [2018-01-25 21:30:57,905] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 20]: expected value 6, actual: 6 [2018-01-25 21:30:57,905] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 20]: expected value 6, actual: 6 [2018-01-25 21:30:57,906] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 21]: expected value 6, actual: 6 [2018-01-25 21:30:57,907] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 21]: expected value 6, actual: 6 [2018-01-25 21:30:57,908] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 22]: expected value 6, actual: 6 [2018-01-25 21:30:57,908] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 22]: expected value 6, actual: 6 [2018-01-25 21:30:57,909] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 23]: expected value 6, actual: 6 [2018-01-25 21:30:57,909] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 23]: expected value 6, actual: 6 [2018-01-25 21:30:57,910] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 24]: expected value 6, actual: 6 [2018-01-25 21:30:57,910] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 24]: expected value 6, actual: 6 [2018-01-25 21:30:57,948] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 25]: expected value 6, actual: 6 [2018-01-25 21:30:57,949] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 25]: expected value 6, actual: 6 [2018-01-25 21:30:57,949] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 26]: expected value 6, actual: 6 [2018-01-25 21:30:57,950] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 26]: expected value 6, actual: 6 [2018-01-25 21:30:57,951] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 27]: expected value 6, actual: 6 [2018-01-25 21:30:57,952] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 27]: expected value 6, actual: 6 [2018-01-25 21:30:57,952] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 28]: expected value 6, actual: 6 [2018-01-25 21:30:57,952] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 9, 28]: expected value 6, actual: 6 [2018-01-25 21:30:57,952] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 1]: expected value 6, actual: 6 [2018-01-25 21:30:57,952] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 1]: expected value 6, actual: 6 [2018-01-25 21:30:57,953] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 2]: expected value 6, actual: 6 [2018-01-25 21:30:57,953] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 2]: expected value 6, actual: 6 [2018-01-25 21:30:57,953] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 3]: expected value 6, actual: 6 [2018-01-25 21:30:57,953] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 3]: expected value 6, actual: 6 [2018-01-25 21:30:57,953] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 4]: expected value 6, actual: 6 [2018-01-25 21:30:57,954] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 4]: expected value 6, actual: 6 [2018-01-25 21:30:57,954] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 5]: expected value 6, actual: 6 [2018-01-25 21:30:57,954] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 5]: expected value 6, actual: 6 [2018-01-25 21:30:57,954] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 6]: expected value 6, actual: 6 [2018-01-25 21:30:57,954] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 6]: expected value 6, actual: 6 [2018-01-25 21:30:57,954] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 7]: expected value 6, actual: 6 [2018-01-25 21:30:57,954] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 7]: expected value 6, actual: 6 [2018-01-25 21:30:57,954] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 8]: expected value 6, actual: 6 [2018-01-25 21:30:57,954] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 8]: expected value 6, actual: 6 [2018-01-25 21:30:57,955] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 9]: expected value 6, actual: 6 [2018-01-25 21:30:57,955] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 9]: expected value 6, actual: 6 [2018-01-25 21:30:57,955] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 10]: expected value 6, actual: 6 [2018-01-25 21:30:57,956] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 10]: expected value 6, actual: 6 [2018-01-25 21:30:57,956] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 11]: expected value 6, actual: 6 [2018-01-25 21:30:57,956] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 11]: expected value 6, actual: 6 [2018-01-25 21:30:57,956] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 12]: expected value 6, actual: 6 [2018-01-25 21:30:57,959] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 12]: expected value 6, actual: 6 [2018-01-25 21:30:57,960] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 13]: expected value 6, actual: 6 [2018-01-25 21:30:57,960] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 13]: expected value 6, actual: 6 [2018-01-25 21:30:57,965] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 14]: expected value 6, actual: 6 [2018-01-25 21:30:57,965] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 14]: expected value 6, actual: 6 [2018-01-25 21:30:57,965] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 15]: expected value 6, actual: 6 [2018-01-25 21:30:57,965] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 15]: expected value 6, actual: 6 [2018-01-25 21:30:57,966] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 16]: expected value 6, actual: 6 [2018-01-25 21:30:57,966] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 16]: expected value 6, actual: 6 [2018-01-25 21:30:57,966] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 17]: expected value 6, actual: 6 [2018-01-25 21:30:57,966] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 17]: expected value 6, actual: 6 [2018-01-25 21:30:57,967] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 18]: expected value 6, actual: 6 [2018-01-25 21:30:57,967] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 18]: expected value 6, actual: 6 [2018-01-25 21:30:57,967] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 19]: expected value 6, actual: 6 [2018-01-25 21:30:57,967] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 19]: expected value 6, actual: 6 [2018-01-25 21:30:57,968] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 20]: expected value 6, actual: 6 [2018-01-25 21:30:57,968] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 20]: expected value 6, actual: 6 [2018-01-25 21:30:57,968] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 21]: expected value 6, actual: 6 [2018-01-25 21:30:57,968] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 21]: expected value 6, actual: 6 [2018-01-25 21:30:57,969] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 22]: expected value 6, actual: 6 [2018-01-25 21:30:57,982] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 22]: expected value 6, actual: 6 [2018-01-25 21:30:57,983] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 23]: expected value 6, actual: 6 [2018-01-25 21:30:57,983] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 23]: expected value 6, actual: 6 [2018-01-25 21:30:57,984] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 24]: expected value 6, actual: 6 [2018-01-25 21:30:57,984] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 24]: expected value 6, actual: 6 [2018-01-25 21:30:57,984] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 25]: expected value 6, actual: 6 [2018-01-25 21:30:57,984] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 25]: expected value 6, actual: 6 [2018-01-25 21:30:57,991] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 26]: expected value 6, actual: 6 [2018-01-25 21:30:57,992] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 26]: expected value 6, actual: 6 [2018-01-25 21:30:57,992] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 27]: expected value 6, actual: 6 [2018-01-25 21:30:57,992] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 27]: expected value 6, actual: 6 [2018-01-25 21:30:57,992] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 28]: expected value 6, actual: 6 [2018-01-25 21:30:57,992] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 10, 28]: expected value 6, actual: 6 [2018-01-25 21:30:57,993] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 1]: expected value 6, actual: 6 [2018-01-25 21:30:57,993] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 1]: expected value 6, actual: 6 [2018-01-25 21:30:57,993] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 2]: expected value 6, actual: 6 [2018-01-25 21:30:57,993] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 2]: expected value 6, actual: 6 [2018-01-25 21:30:57,995] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 3]: expected value 6, actual: 6 [2018-01-25 21:30:57,996] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 3]: expected value 6, actual: 6 [2018-01-25 21:30:57,996] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 4]: expected value 6, actual: 6 [2018-01-25 21:30:57,996] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 4]: expected value 6, actual: 6 [2018-01-25 21:30:57,997] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 5]: expected value 6, actual: 6 [2018-01-25 21:30:58,004] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 5]: expected value 6, actual: 6 [2018-01-25 21:30:58,004] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 6]: expected value 6, actual: 6 [2018-01-25 21:30:58,004] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 6]: expected value 6, actual: 6 [2018-01-25 21:30:58,007] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 7]: expected value 6, actual: 6 [2018-01-25 21:30:58,013] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 7]: expected value 6, actual: 6 [2018-01-25 21:30:58,017] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 8]: expected value 6, actual: 6 [2018-01-25 21:30:58,024] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 8]: expected value 6, actual: 6 [2018-01-25 21:30:58,030] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 9]: expected value 6, actual: 6 [2018-01-25 21:30:58,046] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 9]: expected value 6, actual: 6 [2018-01-25 21:30:58,054] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 10]: expected value 6, actual: 6 [2018-01-25 21:30:58,056] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 10]: expected value 6, actual: 6 [2018-01-25 21:30:58,056] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 11]: expected value 6, actual: 6 [2018-01-25 21:30:58,057] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 11]: expected value 6, actual: 6 [2018-01-25 21:30:58,057] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 12]: expected value 6, actual: 6 [2018-01-25 21:30:58,057] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 12]: expected value 6, actual: 6 [2018-01-25 21:30:58,058] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 13]: expected value 6, actual: 6 [2018-01-25 21:30:58,058] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 13]: expected value 6, actual: 6 [2018-01-25 21:30:58,058] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 14]: expected value 6, actual: 6 [2018-01-25 21:30:58,058] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 14]: expected value 6, actual: 6 [2018-01-25 21:30:58,059] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 15]: expected value 6, actual: 6 [2018-01-25 21:30:58,059] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 15]: expected value 6, actual: 6 [2018-01-25 21:30:58,059] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 16]: expected value 6, actual: 6 [2018-01-25 21:30:58,059] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 16]: expected value 6, actual: 6 [2018-01-25 21:30:58,070] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 17]: expected value 6, actual: 6 [2018-01-25 21:30:58,096] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 17]: expected value 6, actual: 6 [2018-01-25 21:30:58,104] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 18]: expected value 6, actual: 6 [2018-01-25 21:30:58,109] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 18]: expected value 6, actual: 6 [2018-01-25 21:30:58,128] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 19]: expected value 6, actual: 6 [2018-01-25 21:30:58,136] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 19]: expected value 6, actual: 6 [2018-01-25 21:30:58,139] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 20]: expected value 6, actual: 6 [2018-01-25 21:30:58,140] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 20]: expected value 6, actual: 6 [2018-01-25 21:30:58,141] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 21]: expected value 6, actual: 6 [2018-01-25 21:30:58,141] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 21]: expected value 6, actual: 6 [2018-01-25 21:30:58,142] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 22]: expected value 6, actual: 6 [2018-01-25 21:30:58,142] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 22]: expected value 6, actual: 6 [2018-01-25 21:30:58,165] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 23]: expected value 6, actual: 6 [2018-01-25 21:30:58,193] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 23]: expected value 6, actual: 6 [2018-01-25 21:30:58,197] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 24]: expected value 6, actual: 6 [2018-01-25 21:30:58,197] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 24]: expected value 6, actual: 6 [2018-01-25 21:30:58,203] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 25]: expected value 6, actual: 6 [2018-01-25 21:30:58,204] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 25]: expected value 6, actual: 6 [2018-01-25 21:30:58,206] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 26]: expected value 6, actual: 6 [2018-01-25 21:30:58,206] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 26]: expected value 6, actual: 6 [2018-01-25 21:30:58,207] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 27]: expected value 6, actual: 6 [2018-01-25 21:30:58,208] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 27]: expected value 6, actual: 6 [2018-01-25 21:30:58,209] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 28]: expected value 6, actual: 6 [2018-01-25 21:30:58,210] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 11, 28]: expected value 6, actual: 6 [2018-01-25 21:30:58,211] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 1]: expected value 6, actual: 6 [2018-01-25 21:30:58,212] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 1]: expected value 6, actual: 6 [2018-01-25 21:30:58,213] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 2]: expected value 6, actual: 6 [2018-01-25 21:30:58,213] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 2]: expected value 6, actual: 6 [2018-01-25 21:30:58,214] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 3]: expected value 6, actual: 6 [2018-01-25 21:30:58,214] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 3]: expected value 6, actual: 6 [2018-01-25 21:30:58,216] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 4]: expected value 6, actual: 6 [2018-01-25 21:30:58,217] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 4]: expected value 6, actual: 6 [2018-01-25 21:30:58,218] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 5]: expected value 6, actual: 6 [2018-01-25 21:30:58,218] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 5]: expected value 6, actual: 6 [2018-01-25 21:30:58,220] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 6]: expected value 6, actual: 6 [2018-01-25 21:30:58,224] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 6]: expected value 6, actual: 6 [2018-01-25 21:30:58,242] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 7]: expected value 6, actual: 6 [2018-01-25 21:30:58,242] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 7]: expected value 6, actual: 6 [2018-01-25 21:30:58,243] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 8]: expected value 6, actual: 6 [2018-01-25 21:30:58,243] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 8]: expected value 6, actual: 6 [2018-01-25 21:30:58,244] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 9]: expected value 6, actual: 6 [2018-01-25 21:30:58,244] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 9]: expected value 6, actual: 6 [2018-01-25 21:30:58,245] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 10]: expected value 6, actual: 6 [2018-01-25 21:30:58,245] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 10]: expected value 6, actual: 6 [2018-01-25 21:30:58,246] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 11]: expected value 6, actual: 6 [2018-01-25 21:30:58,246] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 11]: expected value 6, actual: 6 [2018-01-25 21:30:58,247] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 12]: expected value 6, actual: 6 [2018-01-25 21:30:58,247] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 12]: expected value 6, actual: 6 [2018-01-25 21:30:58,247] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 13]: expected value 6, actual: 6 [2018-01-25 21:30:58,247] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 13]: expected value 6, actual: 6 [2018-01-25 21:30:58,248] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 14]: expected value 6, actual: 6 [2018-01-25 21:30:58,248] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 14]: expected value 6, actual: 6 [2018-01-25 21:30:58,249] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 15]: expected value 6, actual: 6 [2018-01-25 21:30:58,249] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 15]: expected value 6, actual: 6 [2018-01-25 21:30:58,250] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 16]: expected value 6, actual: 6 [2018-01-25 21:30:58,259] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 16]: expected value 6, actual: 6 [2018-01-25 21:30:58,265] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 17]: expected value 6, actual: 6 [2018-01-25 21:30:58,266] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 17]: expected value 6, actual: 6 [2018-01-25 21:30:58,267] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 18]: expected value 6, actual: 6 [2018-01-25 21:30:58,287] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 18]: expected value 6, actual: 6 [2018-01-25 21:30:58,287] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 19]: expected value 6, actual: 6 [2018-01-25 21:30:58,287] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 19]: expected value 6, actual: 6 [2018-01-25 21:30:58,288] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 20]: expected value 6, actual: 6 [2018-01-25 21:30:58,289] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 20]: expected value 6, actual: 6 [2018-01-25 21:30:58,291] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 21]: expected value 6, actual: 6 [2018-01-25 21:30:58,319] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 21]: expected value 6, actual: 6 [2018-01-25 21:30:58,351] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 22]: expected value 6, actual: 6 [2018-01-25 21:30:58,351] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 22]: expected value 6, actual: 6 [2018-01-25 21:30:58,352] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 23]: expected value 6, actual: 6 [2018-01-25 21:30:58,352] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 23]: expected value 6, actual: 6 [2018-01-25 21:30:58,366] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 24]: expected value 6, actual: 6 [2018-01-25 21:30:58,367] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 24]: expected value 6, actual: 6 [2018-01-25 21:30:58,392] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 25]: expected value 6, actual: 6 [2018-01-25 21:30:58,402] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 25]: expected value 6, actual: 6 [2018-01-25 21:30:58,417] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 26]: expected value 6, actual: 6 [2018-01-25 21:30:58,418] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 26]: expected value 6, actual: 6 [2018-01-25 21:30:58,418] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 27]: expected value 6, actual: 6 [2018-01-25 21:30:58,419] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 27]: expected value 6, actual: 6 [2018-01-25 21:30:58,420] - [task:2816] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 28]: expected value 6, actual: 6 [2018-01-25 21:30:58,420] - [task:2819] INFO - ddoc=test_view-e8b4520, query={'connection_timeout': 60000, 'group_level': '3'}, server=172.17.0.8 Key [2008, 12, 28]: expected value 6, actual: 6 [2018-01-25 21:31:00,286] - [bucket_helper:158] INFO - deleted bucket : default from 172.17.0.8 [2018-01-25 21:31:00,286] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:31:00,293] - [rest_client:133] INFO - node 172.17.0.8 existing buckets : [] [2018-01-25 21:31:00,293] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:31:02,301] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.6 [2018-01-25 21:31:02,301] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:31:04,318] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.7 [2018-01-25 21:31:04,318] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:31:06,328] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.5 [2018-01-25 21:31:06,329] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:31:08,366] - [cluster_helper:254] INFO - rebalancing all nodes in order to remove nodes [2018-01-25 21:31:08,371] - [rest_client:1423] INFO - rebalance params : {'password': 'password', 'ejectedNodes': u'ns_1@172.17.0.7,ns_1@172.17.0.5,ns_1@172.17.0.6', 'user': 'Administrator', 'knownNodes': u'ns_1@cb-example-0003.cb-example.default.svc,ns_1@172.17.0.7,ns_1@172.17.0.5,ns_1@172.17.0.6'} [2018-01-25 21:31:08,390] - [rest_client:1428] INFO - rebalance operation started [2018-01-25 21:31:08,427] - [rest_client:1576] INFO - rebalance percentage : 0.00 % [2018-01-25 21:31:18,448] - [rest_client:1576] INFO - rebalance percentage : 58.33 % [2018-01-25 21:31:28,460] - [rest_client:1576] INFO - rebalance percentage : 58.33 % [2018-01-25 21:31:48,488] - [rest_client:1515] INFO - rebalance progress took 40.0980358124 seconds [2018-01-25 21:31:48,491] - [rest_client:1516] INFO - sleep for 10 seconds after rebalance... [2018-01-25 21:31:58,543] - [cluster_helper:294] INFO - removed all the nodes from cluster associated with ip:172.17.0.8 port:8091 ssh_username:root ? [(u'ns_1@172.17.0.7', 8091), (u'ns_1@172.17.0.5', 8091), (u'ns_1@172.17.0.6', 8091)] [2018-01-25 21:31:58,549] - [cluster_helper:78] INFO - waiting for ns_server @ 172.17.0.8:8091 [2018-01-25 21:31:58,564] - [cluster_helper:80] INFO - ns_server @ 172.17.0.8:8091 is running [2018-01-25 21:31:58,575] - [cluster_helper:78] INFO - waiting for ns_server @ 172.17.0.6:8091 [2018-01-25 21:31:58,579] - [cluster_helper:80] INFO - ns_server @ 172.17.0.6:8091 is running [2018-01-25 21:31:58,584] - [cluster_helper:78] INFO - waiting for ns_server @ 172.17.0.7:8091 [2018-01-25 21:31:58,593] - [cluster_helper:80] INFO - ns_server @ 172.17.0.7:8091 is running [2018-01-25 21:31:58,600] - [cluster_helper:78] INFO - waiting for ns_server @ 172.17.0.5:8091 [2018-01-25 21:31:58,606] - [cluster_helper:80] INFO - ns_server @ 172.17.0.5:8091 is running [2018-01-25 21:31:58,607] - [basetestcase:432] INFO - ============== basetestcase cleanup was finished for test #14 test_employee_dataset_all_queries ============== test_employee_dataset_all_queries (view.viewquerytests.ViewQueryTests) ... ok ---------------------------------------------------------------------- Ran 1 test in 232.389s OK Cluster instance shutdown with force summary so far suite recreatebuckettests.RecreateMembaseBuckets , pass 0 , fail 1 failures so far... recreatebuckettests.RecreateMembaseBuckets.test_default_moxi summary so far suite deletebuckettests.DeleteMembaseBuckets , pass 1 , fail 0 summary so far suite createbuckettests.CreateMembaseBucketsTests , pass 1 , fail 4 failures so far... createbuckettests.CreateMembaseBucketsTests.test_default_moxi createbuckettests.CreateMembaseBucketsTests.test_default_on_non_default_port createbuckettests.CreateMembaseBucketsTests.test_non_default_case_sensitive_different_port createbuckettests.CreateMembaseBucketsTests.test_two_replica summary so far suite setgettests.MembaseBucket , pass 1 , fail 0 summary so far suite expirytests.ExpiryTests , pass 1 , fail 0 summary so far suite memcapable.GetlTests , pass 2 , fail 0 summary so far suite memorysanitytests.MemorySanity , pass 1 , fail 0 summary so far suite drainratetests.DrainRateTests , pass 1 , fail 0 summary so far suite view.viewquerytests.ViewQueryTests , pass 1 , fail 0 testrunner logs, diags and results are available under /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_14 Logs will be stored at /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_15 ./testrunner -i ./4node.ini -t view.createdeleteview.CreateDeleteViewTests.test_view_ops,ddoc_ops=update,test_with_view=True,num_ddocs=2,num_views_per_ddoc=3,items=1000,sasl_buckets=1,standard_buckets=1 Test Input params: {'ddoc_ops': 'update', 'num_ddocs': '2', 'items': '1000', 'test_with_view': 'True', 'num_views_per_ddoc': '3', 'num_nodes': 4, 'cluster_name': '4node', 'standard_buckets': '1', 'ini': './4node.ini', 'case_number': 15, 'sasl_buckets': '1', 'conf_file': './4node.conf', 'logs_folder': '/testrunner/logs/testrunner-18-Jan-25_21-16-28/test_15', 'spec': '4node'} [2018-01-25 21:31:58,663] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 21:31:58,797] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 21:31:59,350] - [rest_client:1964] INFO - Node version in cluster 5.0.1-5003-enterprise [2018-01-25 21:31:59,364] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise'] [2018-01-25 21:31:59,364] - [basetestcase:201] INFO - ============== basetestcase setup was started for test #15 test_view_ops============== [2018-01-25 21:31:59,386] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.8 [2018-01-25 21:31:59,386] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:32:01,396] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.6 [2018-01-25 21:32:01,397] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:32:03,412] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.7 [2018-01-25 21:32:03,413] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:32:05,421] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.5 [2018-01-25 21:32:05,421] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:32:07,454] - [basetestcase:482] INFO - sleep for 10 secs. ... [2018-01-25 21:32:17,463] - [cluster_helper:78] INFO - waiting for ns_server @ 172.17.0.8:8091 [2018-01-25 21:32:17,468] - [cluster_helper:80] INFO - ns_server @ 172.17.0.8:8091 is running [2018-01-25 21:32:17,474] - [cluster_helper:78] INFO - waiting for ns_server @ 172.17.0.6:8091 [2018-01-25 21:32:17,478] - [cluster_helper:80] INFO - ns_server @ 172.17.0.6:8091 is running [2018-01-25 21:32:17,484] - [cluster_helper:78] INFO - waiting for ns_server @ 172.17.0.7:8091 [2018-01-25 21:32:17,487] - [cluster_helper:80] INFO - ns_server @ 172.17.0.7:8091 is running [2018-01-25 21:32:17,501] - [cluster_helper:78] INFO - waiting for ns_server @ 172.17.0.5:8091 [2018-01-25 21:32:17,504] - [cluster_helper:80] INFO - ns_server @ 172.17.0.5:8091 is running [2018-01-25 21:32:17,505] - [basetestcase:245] INFO - initializing cluster [2018-01-25 21:32:17,691] - [task:125] INFO - server: ip:172.17.0.8 port:8091 ssh_username:root, nodes/self: {'ip': u'cb-example-0003.cb-example.default.svc', 'availableStorage': [], 'rest_username': '', 'id': u'ns_1@cb-example-0003.cb-example.default.svc', 'uptime': u'1182', 'mcdMemoryReserved': 6234, 'storageTotalRam': 7793, 'hostname': u'cb-example-0003.cb-example.default.svc:8091', 'storage': [], 'moxi': 11211, 'port': u'8091', 'version': u'5.0.1-5003-enterprise', 'memcached': 11210, 'status': u'healthy', 'clusterCompatibility': 327680, 'curr_items': 0, 'services': [u'fts', u'index', u'kv', u'n1ql'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 5708701696, 'memoryTotal': 8171810816, 'memoryQuota': 3900, 'mcdMemoryAllocated': 6234, 'os': u'x86_64-unknown-linux-gnu', 'ports': []} [2018-01-25 21:32:17,692] - [task:157] INFO - quota for index service will be 256 MB [2018-01-25 21:32:17,692] - [task:159] INFO - set index quota to node 172.17.0.8 [2018-01-25 21:32:17,692] - [rest_client:919] INFO - pools/default params : indexMemoryQuota=256 [2018-01-25 21:32:17,698] - [rest_client:907] INFO - pools/default params : memoryQuota=3900 [2018-01-25 21:32:17,708] - [rest_client:942] INFO - settings/indexes params : storageMode=plasma [2018-01-25 21:32:17,715] - [rest_client:807] ERROR - POST http://172.17.0.8:8091/settings/indexes body: storageMode=plasma headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 400 reason: unknown {"errors":{"storageMode":"Changing the optimization mode of global indexes is not supported when index service nodes are present in the cluster. Please remove all index service nodes to change this option."}} auth: Administrator:password [2018-01-25 21:32:17,715] - [rest_client:885] INFO - /node/controller/setupServices params on 172.17.0.8: 8091:services=n1ql%2Ckv%2Cindex&password=password&hostname=172.17.0.8&user=Administrator [2018-01-25 21:32:17,735] - [rest_client:807] ERROR - POST http://172.17.0.8:8091//node/controller/setupServices body: services=n1ql%2Ckv%2Cindex&password=password&hostname=172.17.0.8&user=Administrator headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 400 reason: unknown ["cannot change node services after cluster is provisioned"] auth: Administrator:password [2018-01-25 21:32:17,736] - [rest_client:890] INFO - This node is already provisioned with services, we do not consider this as failure for test case [2018-01-25 21:32:17,737] - [rest_client:825] INFO - settings/web params on 172.17.0.8:8091:username=Administrator&password=password&port=8091 [2018-01-25 21:32:17,837] - [task:125] INFO - server: ip:172.17.0.6 port:8091 ssh_username:root, nodes/self: {'ip': u'172.17.0.6', 'availableStorage': [], 'rest_username': '', 'id': u'ns_1@172.17.0.6', 'uptime': u'1208', 'mcdMemoryReserved': 6234, 'storageTotalRam': 7793, 'hostname': u'172.17.0.6:8091', 'storage': [], 'moxi': 11211, 'port': u'8091', 'version': u'5.0.1-5003-enterprise', 'memcached': 11210, 'status': u'healthy', 'clusterCompatibility': 327680, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 5707522048, 'memoryTotal': 8171810816, 'memoryQuota': 4675, 'mcdMemoryAllocated': 6234, 'os': u'x86_64-unknown-linux-gnu', 'ports': []} [2018-01-25 21:32:17,838] - [rest_client:907] INFO - pools/default params : memoryQuota=4156 [2018-01-25 21:32:17,842] - [rest_client:942] INFO - settings/indexes params : storageMode=plasma [2018-01-25 21:32:17,849] - [rest_client:825] INFO - settings/web params on 172.17.0.6:8091:username=Administrator&password=password&port=8091 [2018-01-25 21:32:17,989] - [task:125] INFO - server: ip:172.17.0.7 port:8091 ssh_username:root, nodes/self: {'ip': u'172.17.0.7', 'availableStorage': [], 'rest_username': '', 'id': u'ns_1@172.17.0.7', 'uptime': u'1197', 'mcdMemoryReserved': 6234, 'storageTotalRam': 7793, 'hostname': u'172.17.0.7:8091', 'storage': [], 'moxi': 11211, 'port': u'8091', 'version': u'5.0.1-5003-enterprise', 'memcached': 11210, 'status': u'healthy', 'clusterCompatibility': 327680, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 5707988992, 'memoryTotal': 8171810816, 'memoryQuota': 4675, 'mcdMemoryAllocated': 6234, 'os': u'x86_64-unknown-linux-gnu', 'ports': []} [2018-01-25 21:32:17,989] - [rest_client:907] INFO - pools/default params : memoryQuota=4156 [2018-01-25 21:32:17,997] - [rest_client:942] INFO - settings/indexes params : storageMode=plasma [2018-01-25 21:32:18,004] - [rest_client:825] INFO - settings/web params on 172.17.0.7:8091:username=Administrator&password=password&port=8091 [2018-01-25 21:32:18,092] - [task:125] INFO - server: ip:172.17.0.5 port:8091 ssh_username:root, nodes/self: {'ip': u'172.17.0.5', 'availableStorage': [], 'rest_username': '', 'id': u'ns_1@172.17.0.5', 'uptime': u'1235', 'mcdMemoryReserved': 6234, 'storageTotalRam': 7793, 'hostname': u'172.17.0.5:8091', 'storage': [], 'moxi': 11211, 'port': u'8091', 'version': u'5.0.1-5003-enterprise', 'memcached': 11210, 'status': u'healthy', 'clusterCompatibility': 327680, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 5707231232, 'memoryTotal': 8171810816, 'memoryQuota': 4675, 'mcdMemoryAllocated': 6234, 'os': u'x86_64-unknown-linux-gnu', 'ports': []} [2018-01-25 21:32:18,095] - [rest_client:907] INFO - pools/default params : memoryQuota=4156 [2018-01-25 21:32:18,102] - [rest_client:942] INFO - settings/indexes params : storageMode=plasma [2018-01-25 21:32:18,113] - [rest_client:825] INFO - settings/web params on 172.17.0.5:8091:username=Administrator&password=password&port=8091 [2018-01-25 21:32:18,221] - [basetestcase:2060] INFO - **** add built-in 'cbadminbucket' user to node 172.17.0.8 **** [2018-01-25 21:32:18,364] - [basetestcase:482] INFO - sleep for 10 secs. ... [2018-01-25 21:32:28,373] - [basetestcase:2065] INFO - **** add 'admin' role to 'cbadminbucket' user **** [2018-01-25 21:32:28,389] - [basetestcase:482] INFO - sleep for 10 secs. ... [2018-01-25 21:32:38,396] - [basetestcase:269] INFO - done initializing cluster [2018-01-25 21:32:39,234] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise'] [2018-01-25 21:32:39,235] - [rest_client:2237] INFO - http://172.17.0.8:8091/pools/default/buckets with param: bucketType=membase&threadsNumber=3&authType=sasl&replicaIndex=1&name=default&saslPassword=None&evictionPolicy=valueOnly&flushEnabled=1&replicaNumber=1&ramQuotaMB=1300 [2018-01-25 21:32:39,242] - [rest_client:2262] INFO - 0.01 seconds to create bucket default [2018-01-25 21:32:39,243] - [bucket_helper:343] INFO - waiting for memcached bucket : default in 172.17.0.8 to accept set ops [2018-01-25 21:32:40,490] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:32:40,702] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:32:41,614] - [task:312] INFO - bucket 'default' was created with per node RAM quota: 1300 [2018-01-25 21:32:42,631] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise'] [2018-01-25 21:32:42,631] - [rest_client:2237] INFO - http://172.17.0.8:8091/pools/default/buckets with param: bucketType=membase&threadsNumber=3&authType=sasl&replicaIndex=1&name=bucket0&saslPassword=password&evictionPolicy=valueOnly&flushEnabled=1&replicaNumber=1&ramQuotaMB=1300 [2018-01-25 21:32:42,636] - [rest_client:2262] INFO - 0.00 seconds to create bucket bucket0 [2018-01-25 21:32:42,636] - [bucket_helper:343] INFO - waiting for memcached bucket : bucket0 in 172.17.0.8 to accept set ops [2018-01-25 21:32:43,884] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 bucket0 [2018-01-25 21:32:44,072] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 bucket0 [2018-01-25 21:32:45,772] - [task:312] INFO - bucket 'bucket0' was created with per node RAM quota: 1300 [2018-01-25 21:32:46,789] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise'] [2018-01-25 21:32:46,789] - [rest_client:2237] INFO - http://172.17.0.8:8091/pools/default/buckets with param: bucketType=membase&threadsNumber=3&authType=none&replicaIndex=1&name=standard_bucket0&evictionPolicy=valueOnly&flushEnabled=1&replicaNumber=1&ramQuotaMB=1300 [2018-01-25 21:32:46,794] - [rest_client:2262] INFO - 0.00 seconds to create bucket standard_bucket0 [2018-01-25 21:32:46,794] - [bucket_helper:343] INFO - waiting for memcached bucket : standard_bucket0 in 172.17.0.8 to accept set ops [2018-01-25 21:32:48,049] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 standard_bucket0 [2018-01-25 21:32:48,728] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 standard_bucket0 [2018-01-25 21:32:50,499] - [task:312] INFO - bucket 'standard_bucket0' was created with per node RAM quota: 1300 [2018-01-25 21:32:50,546] - [basetestcase:344] INFO - ============== basetestcase setup was finished for test #15 test_view_ops ============== [2018-01-25 21:32:50,554] - [basetestcase:482] INFO - sleep for 10 secs. ... [2018-01-25 21:33:00,557] - [basetestcase:1114] INFO - create 1000 documents... [2018-01-25 21:33:00,648] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:33:00,812] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 bucket0 [2018-01-25 21:33:00,947] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 standard_bucket0 [2018-01-25 21:33:01,662] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 bucket0 [2018-01-25 21:33:01,709] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 standard_bucket0 [2018-01-25 21:33:01,711] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:33:01,717] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 bucket0 [2018-01-25 21:33:02,184] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:33:02,324] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 standard_bucket0 [2018-01-25 21:33:02,337] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 bucket0 [2018-01-25 21:33:02,491] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:33:02,970] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 bucket0 [2018-01-25 21:33:03,001] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 standard_bucket0 [2018-01-25 21:33:03,174] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:33:03,489] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 standard_bucket0 [2018-01-25 21:33:03,816] - [createdeleteview:78] INFO - Processing Create DDoc Operation On Bucket default [2018-01-25 21:33:04,535] - [rest_client:807] ERROR - GET http://cb-example-0003.cb-example.default.svc:8092//default/_design/dev_ddoc0 body: headers: {'Content-Type': 'application/json', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 404 reason: not_found {"error":"not_found","reason":"missing"} auth: Administrator:password [2018-01-25 21:33:04,567] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/dev_ddoc0/_view/views0?stale=ok [2018-01-25 21:33:09,618] - [task:2181] INFO - view : views0 was created successfully in ddoc: dev_ddoc0 [2018-01-25 21:33:10,686] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/dev_ddoc0/_view/views1?stale=ok [2018-01-25 21:33:10,716] - [rest_client:807] ERROR - GET http://cb-example-0003.cb-example.default.svc:8092/default/_design/dev_ddoc0/_view/views1?stale=ok body: headers: {'Content-Type': 'application/json', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 500 reason: unknown_error {"error":"unknown_error","reason":"view_undefined"} auth: Administrator:password [2018-01-25 21:33:12,718] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/dev_ddoc0/_view/views1?stale=ok [2018-01-25 21:33:12,725] - [task:2181] INFO - view : views1 was created successfully in ddoc: dev_ddoc0 [2018-01-25 21:33:13,810] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/dev_ddoc0/_view/views2?stale=ok [2018-01-25 21:33:18,845] - [task:2181] INFO - view : views2 was created successfully in ddoc: dev_ddoc0 [2018-01-25 21:33:19,860] - [rest_client:807] ERROR - GET http://cb-example-0003.cb-example.default.svc:8092//default/_design/dev_ddoc1 body: headers: {'Content-Type': 'application/json', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 404 reason: not_found {"error":"not_found","reason":"missing"} auth: Administrator:password [2018-01-25 21:33:19,891] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/dev_ddoc1/_view/views0?stale=ok [2018-01-25 21:33:24,966] - [task:2181] INFO - view : views0 was created successfully in ddoc: dev_ddoc1 [2018-01-25 21:33:26,054] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/dev_ddoc1/_view/views1?stale=ok [2018-01-25 21:33:31,104] - [task:2181] INFO - view : views1 was created successfully in ddoc: dev_ddoc1 [2018-01-25 21:33:32,212] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/dev_ddoc1/_view/views2?stale=ok [2018-01-25 21:33:32,220] - [task:2181] INFO - view : views2 was created successfully in ddoc: dev_ddoc1 [2018-01-25 21:33:32,243] - [createdeleteview:95] INFO - Processing Update DDoc Operation On Bucket default [2018-01-25 21:33:33,333] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/dev_ddoc1/_view/views0?stale=ok [2018-01-25 21:33:38,376] - [task:2181] INFO - view : views0 was created successfully in ddoc: dev_ddoc1 [2018-01-25 21:33:38,418] - [createdeleteview:78] INFO - Processing Create DDoc Operation On Bucket bucket0 [2018-01-25 21:33:39,397] - [rest_client:807] ERROR - GET http://cb-example-0003.cb-example.default.svc:8092//bucket0/_design/dev_ddoc0 body: headers: {'Content-Type': 'application/json', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 404 reason: not_found {"error":"not_found","reason":"missing"} auth: Administrator:password [2018-01-25 21:33:39,443] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/bucket0/_design/dev_ddoc0/_view/views0?stale=ok [2018-01-25 21:33:44,486] - [task:2181] INFO - view : views0 was created successfully in ddoc: dev_ddoc0 [2018-01-25 21:33:45,563] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/bucket0/_design/dev_ddoc0/_view/views1?stale=ok [2018-01-25 21:33:50,630] - [task:2181] INFO - view : views1 was created successfully in ddoc: dev_ddoc0 [2018-01-25 21:33:51,734] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/bucket0/_design/dev_ddoc0/_view/views2?stale=ok [2018-01-25 21:33:51,743] - [rest_client:807] ERROR - GET http://cb-example-0003.cb-example.default.svc:8092/bucket0/_design/dev_ddoc0/_view/views2?stale=ok body: headers: {'Content-Type': 'application/json', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 500 reason: unknown_error {"error":"unknown_error","reason":"view_undefined"} auth: Administrator:password [2018-01-25 21:33:53,745] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/bucket0/_design/dev_ddoc0/_view/views2?stale=ok [2018-01-25 21:33:53,757] - [task:2181] INFO - view : views2 was created successfully in ddoc: dev_ddoc0 [2018-01-25 21:33:54,785] - [rest_client:807] ERROR - GET http://cb-example-0003.cb-example.default.svc:8092//bucket0/_design/dev_ddoc1 body: headers: {'Content-Type': 'application/json', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 404 reason: not_found {"error":"not_found","reason":"missing"} auth: Administrator:password [2018-01-25 21:33:54,833] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/bucket0/_design/dev_ddoc1/_view/views0?stale=ok [2018-01-25 21:33:59,887] - [task:2181] INFO - view : views0 was created successfully in ddoc: dev_ddoc1 [2018-01-25 21:34:00,961] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/bucket0/_design/dev_ddoc1/_view/views1?stale=ok [2018-01-25 21:34:06,002] - [task:2181] INFO - view : views1 was created successfully in ddoc: dev_ddoc1 [2018-01-25 21:34:07,092] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/bucket0/_design/dev_ddoc1/_view/views2?stale=ok [2018-01-25 21:34:07,099] - [task:2181] INFO - view : views2 was created successfully in ddoc: dev_ddoc1 [2018-01-25 21:34:07,113] - [createdeleteview:95] INFO - Processing Update DDoc Operation On Bucket bucket0 [2018-01-25 21:34:08,277] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/bucket0/_design/dev_ddoc1/_view/views0?stale=ok [2018-01-25 21:34:08,292] - [rest_client:807] ERROR - GET http://cb-example-0003.cb-example.default.svc:8092/bucket0/_design/dev_ddoc1/_view/views0?stale=ok body: headers: {'Content-Type': 'application/json', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 500 reason: unknown_error {"error":"unknown_error","reason":"view_undefined"} auth: Administrator:password [2018-01-25 21:34:10,295] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/bucket0/_design/dev_ddoc1/_view/views0?stale=ok [2018-01-25 21:34:10,304] - [task:2181] INFO - view : views0 was created successfully in ddoc: dev_ddoc1 [2018-01-25 21:34:10,328] - [createdeleteview:78] INFO - Processing Create DDoc Operation On Bucket standard_bucket0 [2018-01-25 21:34:11,325] - [rest_client:807] ERROR - GET http://cb-example-0003.cb-example.default.svc:8092//standard_bucket0/_design/dev_ddoc0 body: headers: {'Content-Type': 'application/json', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 404 reason: not_found {"error":"not_found","reason":"missing"} auth: Administrator:password [2018-01-25 21:34:11,356] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/standard_bucket0/_design/dev_ddoc0/_view/views0?stale=ok [2018-01-25 21:34:11,385] - [rest_client:807] ERROR - GET http://cb-example-0003.cb-example.default.svc:8092/standard_bucket0/_design/dev_ddoc0/_view/views0?stale=ok body: headers: {'Content-Type': 'application/json', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 500 reason: unknown_error {"error":"unknown_error","reason":"view_undefined"} auth: Administrator:password [2018-01-25 21:34:13,388] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/standard_bucket0/_design/dev_ddoc0/_view/views0?stale=ok [2018-01-25 21:34:13,394] - [task:2181] INFO - view : views0 was created successfully in ddoc: dev_ddoc0 [2018-01-25 21:34:14,483] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/standard_bucket0/_design/dev_ddoc0/_view/views1?stale=ok [2018-01-25 21:34:19,547] - [task:2181] INFO - view : views1 was created successfully in ddoc: dev_ddoc0 [2018-01-25 21:34:20,631] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/standard_bucket0/_design/dev_ddoc0/_view/views2?stale=ok [2018-01-25 21:34:20,638] - [rest_client:807] ERROR - GET http://cb-example-0003.cb-example.default.svc:8092/standard_bucket0/_design/dev_ddoc0/_view/views2?stale=ok body: headers: {'Content-Type': 'application/json', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 500 reason: unknown_error {"error":"unknown_error","reason":"view_undefined"} auth: Administrator:password [2018-01-25 21:34:22,641] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/standard_bucket0/_design/dev_ddoc0/_view/views2?stale=ok [2018-01-25 21:34:22,649] - [task:2181] INFO - view : views2 was created successfully in ddoc: dev_ddoc0 [2018-01-25 21:34:23,672] - [rest_client:807] ERROR - GET http://cb-example-0003.cb-example.default.svc:8092//standard_bucket0/_design/dev_ddoc1 body: headers: {'Content-Type': 'application/json', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 404 reason: not_found {"error":"not_found","reason":"missing"} auth: Administrator:password [2018-01-25 21:34:23,705] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/standard_bucket0/_design/dev_ddoc1/_view/views0?stale=ok [2018-01-25 21:34:28,758] - [task:2181] INFO - view : views0 was created successfully in ddoc: dev_ddoc1 [2018-01-25 21:34:29,837] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/standard_bucket0/_design/dev_ddoc1/_view/views1?stale=ok [2018-01-25 21:34:29,881] - [rest_client:807] ERROR - GET http://cb-example-0003.cb-example.default.svc:8092/standard_bucket0/_design/dev_ddoc1/_view/views1?stale=ok body: headers: {'Content-Type': 'application/json', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 500 reason: unknown_error {"error":"unknown_error","reason":"view_undefined"} auth: Administrator:password [2018-01-25 21:34:31,885] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/standard_bucket0/_design/dev_ddoc1/_view/views1?stale=ok [2018-01-25 21:34:31,892] - [task:2181] INFO - view : views1 was created successfully in ddoc: dev_ddoc1 [2018-01-25 21:34:33,005] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/standard_bucket0/_design/dev_ddoc1/_view/views2?stale=ok [2018-01-25 21:34:33,016] - [task:2181] INFO - view : views2 was created successfully in ddoc: dev_ddoc1 [2018-01-25 21:34:33,066] - [createdeleteview:95] INFO - Processing Update DDoc Operation On Bucket standard_bucket0 [2018-01-25 21:34:34,122] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/standard_bucket0/_design/dev_ddoc1/_view/views0?stale=ok [2018-01-25 21:34:39,142] - [task:2181] INFO - view : views0 was created successfully in ddoc: dev_ddoc1 [2018-01-25 21:34:39,197] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise'] [2018-01-25 21:34:39,203] - [basetestcase:2242] INFO - list of all nodes in cluster: [] [2018-01-25 21:34:39,203] - [createdeleteview:241] INFO - DDoc Validation Started [2018-01-25 21:34:39,212] - [createdeleteview:249] INFO - Database Document dev_ddoc1 details : {"views": {"views0": {"map": "function (doc) { emit(null, doc);}"}, "views1": {"map": "function (doc) { emit(doc.age, doc.first_name);}"}, "views2": {"map": "function (doc) { emit(doc.age, doc.first_name);}"}}} [2018-01-25 21:34:39,215] - [createdeleteview:249] INFO - Database Document dev_ddoc0 details : {"views": {"views0": {"map": "function (doc) { emit(doc.age, doc.first_name);}"}, "views1": {"map": "function (doc) { emit(doc.age, doc.first_name);}"}, "views2": {"map": "function (doc) { emit(doc.age, doc.first_name);}"}}} [2018-01-25 21:34:39,220] - [createdeleteview:249] INFO - Database Document dev_ddoc1 details : {"views": {"views0": {"map": "function (doc) { emit(null, doc);}"}, "views1": {"map": "function (doc) { emit(doc.age, doc.first_name);}"}, "views2": {"map": "function (doc) { emit(doc.age, doc.first_name);}"}}} [2018-01-25 21:34:39,224] - [createdeleteview:249] INFO - Database Document dev_ddoc0 details : {"views": {"views0": {"map": "function (doc) { emit(doc.age, doc.first_name);}"}, "views1": {"map": "function (doc) { emit(doc.age, doc.first_name);}"}, "views2": {"map": "function (doc) { emit(doc.age, doc.first_name);}"}}} [2018-01-25 21:34:39,230] - [createdeleteview:249] INFO - Database Document dev_ddoc1 details : {"views": {"views0": {"map": "function (doc) { emit(null, doc);}"}, "views1": {"map": "function (doc) { emit(doc.age, doc.first_name);}"}, "views2": {"map": "function (doc) { emit(doc.age, doc.first_name);}"}}} [2018-01-25 21:34:39,237] - [createdeleteview:249] INFO - Database Document dev_ddoc0 details : {"views": {"views0": {"map": "function (doc) { emit(doc.age, doc.first_name);}"}, "views1": {"map": "function (doc) { emit(doc.age, doc.first_name);}"}, "views2": {"map": "function (doc) { emit(doc.age, doc.first_name);}"}}} [2018-01-25 21:34:39,237] - [createdeleteview:258] INFO - DDoc Validation Successful [2018-01-25 21:34:39,248] - [createdeleteview:272] INFO - DDoc Data Validation Started on bucket bucket0. Expected Data Items 1000 [2018-01-25 21:34:40,164] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/bucket0/_design/dev_ddoc1/_view/views0?stale=false&connection_timeout=60000&full_set=true [2018-01-25 21:34:41,103] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/bucket0/_design/dev_ddoc1/_view/views0?stale=false&connection_timeout=60000&full_set=true [2018-01-25 21:34:41,142] - [task:2384] INFO - Server: 172.17.0.8, Design Doc: dev_ddoc1, View: views0, (1000 rows) expected, (1000 rows) returned [2018-01-25 21:34:41,143] - [task:2391] INFO - expected number of rows: '1000' was found for view query [2018-01-25 21:34:42,150] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/bucket0/_design/dev_ddoc1/_view/views1?stale=false&connection_timeout=60000&full_set=true [2018-01-25 21:34:42,169] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/bucket0/_design/dev_ddoc1/_view/views1?stale=false&connection_timeout=60000&full_set=true [2018-01-25 21:34:42,184] - [task:2384] INFO - Server: 172.17.0.8, Design Doc: dev_ddoc1, View: views1, (1000 rows) expected, (1000 rows) returned [2018-01-25 21:34:42,185] - [task:2391] INFO - expected number of rows: '1000' was found for view query [2018-01-25 21:34:43,191] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/bucket0/_design/dev_ddoc1/_view/views2?stale=false&connection_timeout=60000&full_set=true [2018-01-25 21:34:43,207] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/bucket0/_design/dev_ddoc1/_view/views2?stale=false&connection_timeout=60000&full_set=true [2018-01-25 21:34:43,219] - [task:2384] INFO - Server: 172.17.0.8, Design Doc: dev_ddoc1, View: views2, (1000 rows) expected, (1000 rows) returned [2018-01-25 21:34:43,219] - [task:2391] INFO - expected number of rows: '1000' was found for view query [2018-01-25 21:34:44,225] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/bucket0/_design/dev_ddoc0/_view/views0?stale=false&connection_timeout=60000&full_set=true [2018-01-25 21:34:45,167] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/bucket0/_design/dev_ddoc0/_view/views0?stale=false&connection_timeout=60000&full_set=true [2018-01-25 21:34:45,184] - [task:2384] INFO - Server: 172.17.0.8, Design Doc: dev_ddoc0, View: views0, (1000 rows) expected, (1000 rows) returned [2018-01-25 21:34:45,184] - [task:2391] INFO - expected number of rows: '1000' was found for view query [2018-01-25 21:34:46,193] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/bucket0/_design/dev_ddoc0/_view/views1?stale=false&connection_timeout=60000&full_set=true [2018-01-25 21:34:46,211] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/bucket0/_design/dev_ddoc0/_view/views1?stale=false&connection_timeout=60000&full_set=true [2018-01-25 21:34:46,228] - [task:2384] INFO - Server: 172.17.0.8, Design Doc: dev_ddoc0, View: views1, (1000 rows) expected, (1000 rows) returned [2018-01-25 21:34:46,228] - [task:2391] INFO - expected number of rows: '1000' was found for view query [2018-01-25 21:34:47,234] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/bucket0/_design/dev_ddoc0/_view/views2?stale=false&connection_timeout=60000&full_set=true [2018-01-25 21:34:47,258] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/bucket0/_design/dev_ddoc0/_view/views2?stale=false&connection_timeout=60000&full_set=true [2018-01-25 21:34:47,273] - [task:2384] INFO - Server: 172.17.0.8, Design Doc: dev_ddoc0, View: views2, (1000 rows) expected, (1000 rows) returned [2018-01-25 21:34:47,273] - [task:2391] INFO - expected number of rows: '1000' was found for view query [2018-01-25 21:34:47,274] - [createdeleteview:272] INFO - DDoc Data Validation Started on bucket default. Expected Data Items 1000 [2018-01-25 21:34:48,280] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/dev_ddoc1/_view/views0?stale=false&connection_timeout=60000&full_set=true [2018-01-25 21:34:49,291] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/dev_ddoc1/_view/views0?stale=false&connection_timeout=60000&full_set=true [2018-01-25 21:34:49,315] - [task:2384] INFO - Server: 172.17.0.8, Design Doc: dev_ddoc1, View: views0, (1000 rows) expected, (1000 rows) returned [2018-01-25 21:34:49,316] - [task:2391] INFO - expected number of rows: '1000' was found for view query [2018-01-25 21:34:50,323] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/dev_ddoc1/_view/views1?stale=false&connection_timeout=60000&full_set=true [2018-01-25 21:34:50,345] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/dev_ddoc1/_view/views1?stale=false&connection_timeout=60000&full_set=true [2018-01-25 21:34:50,362] - [task:2384] INFO - Server: 172.17.0.8, Design Doc: dev_ddoc1, View: views1, (1000 rows) expected, (1000 rows) returned [2018-01-25 21:34:50,362] - [task:2391] INFO - expected number of rows: '1000' was found for view query [2018-01-25 21:34:51,369] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/dev_ddoc1/_view/views2?stale=false&connection_timeout=60000&full_set=true [2018-01-25 21:34:51,389] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/dev_ddoc1/_view/views2?stale=false&connection_timeout=60000&full_set=true [2018-01-25 21:34:51,405] - [task:2384] INFO - Server: 172.17.0.8, Design Doc: dev_ddoc1, View: views2, (1000 rows) expected, (1000 rows) returned [2018-01-25 21:34:51,406] - [task:2391] INFO - expected number of rows: '1000' was found for view query [2018-01-25 21:34:51,410] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/dev_ddoc0/_view/views0?stale=false&connection_timeout=60000&full_set=true [2018-01-25 21:34:52,477] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/dev_ddoc0/_view/views0?stale=false&connection_timeout=60000&full_set=true [2018-01-25 21:34:52,490] - [task:2384] INFO - Server: 172.17.0.8, Design Doc: dev_ddoc0, View: views0, (1000 rows) expected, (1000 rows) returned [2018-01-25 21:34:52,491] - [task:2391] INFO - expected number of rows: '1000' was found for view query [2018-01-25 21:34:52,496] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/dev_ddoc0/_view/views1?stale=false&connection_timeout=60000&full_set=true [2018-01-25 21:34:52,518] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/dev_ddoc0/_view/views1?stale=false&connection_timeout=60000&full_set=true [2018-01-25 21:34:52,534] - [task:2384] INFO - Server: 172.17.0.8, Design Doc: dev_ddoc0, View: views1, (1000 rows) expected, (1000 rows) returned [2018-01-25 21:34:52,534] - [task:2391] INFO - expected number of rows: '1000' was found for view query [2018-01-25 21:34:52,541] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/dev_ddoc0/_view/views2?stale=false&connection_timeout=60000&full_set=true [2018-01-25 21:34:52,603] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/default/_design/dev_ddoc0/_view/views2?stale=false&connection_timeout=60000&full_set=true [2018-01-25 21:34:52,621] - [task:2384] INFO - Server: 172.17.0.8, Design Doc: dev_ddoc0, View: views2, (1000 rows) expected, (1000 rows) returned [2018-01-25 21:34:52,621] - [task:2391] INFO - expected number of rows: '1000' was found for view query [2018-01-25 21:34:52,622] - [createdeleteview:272] INFO - DDoc Data Validation Started on bucket standard_bucket0. Expected Data Items 1000 [2018-01-25 21:34:52,629] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/standard_bucket0/_design/dev_ddoc1/_view/views0?stale=false&connection_timeout=60000&full_set=true [2018-01-25 21:34:53,677] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/standard_bucket0/_design/dev_ddoc1/_view/views0?stale=false&connection_timeout=60000&full_set=true [2018-01-25 21:34:53,701] - [task:2384] INFO - Server: 172.17.0.8, Design Doc: dev_ddoc1, View: views0, (1000 rows) expected, (1000 rows) returned [2018-01-25 21:34:53,702] - [task:2391] INFO - expected number of rows: '1000' was found for view query [2018-01-25 21:34:54,712] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/standard_bucket0/_design/dev_ddoc1/_view/views1?stale=false&connection_timeout=60000&full_set=true [2018-01-25 21:34:54,737] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/standard_bucket0/_design/dev_ddoc1/_view/views1?stale=false&connection_timeout=60000&full_set=true [2018-01-25 21:34:54,764] - [task:2384] INFO - Server: 172.17.0.8, Design Doc: dev_ddoc1, View: views1, (1000 rows) expected, (1000 rows) returned [2018-01-25 21:34:54,764] - [task:2391] INFO - expected number of rows: '1000' was found for view query [2018-01-25 21:34:54,774] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/standard_bucket0/_design/dev_ddoc1/_view/views2?stale=false&connection_timeout=60000&full_set=true [2018-01-25 21:34:54,804] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/standard_bucket0/_design/dev_ddoc1/_view/views2?stale=false&connection_timeout=60000&full_set=true [2018-01-25 21:34:54,823] - [task:2384] INFO - Server: 172.17.0.8, Design Doc: dev_ddoc1, View: views2, (1000 rows) expected, (1000 rows) returned [2018-01-25 21:34:54,824] - [task:2391] INFO - expected number of rows: '1000' was found for view query [2018-01-25 21:34:54,833] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/standard_bucket0/_design/dev_ddoc0/_view/views0?stale=false&connection_timeout=60000&full_set=true [2018-01-25 21:34:55,794] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/standard_bucket0/_design/dev_ddoc0/_view/views0?stale=false&connection_timeout=60000&full_set=true [2018-01-25 21:34:55,811] - [task:2384] INFO - Server: 172.17.0.8, Design Doc: dev_ddoc0, View: views0, (1000 rows) expected, (1000 rows) returned [2018-01-25 21:34:55,812] - [task:2391] INFO - expected number of rows: '1000' was found for view query [2018-01-25 21:34:55,817] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/standard_bucket0/_design/dev_ddoc0/_view/views1?stale=false&connection_timeout=60000&full_set=true [2018-01-25 21:34:55,837] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/standard_bucket0/_design/dev_ddoc0/_view/views1?stale=false&connection_timeout=60000&full_set=true [2018-01-25 21:34:55,853] - [task:2384] INFO - Server: 172.17.0.8, Design Doc: dev_ddoc0, View: views1, (1000 rows) expected, (1000 rows) returned [2018-01-25 21:34:55,853] - [task:2391] INFO - expected number of rows: '1000' was found for view query [2018-01-25 21:34:55,862] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/standard_bucket0/_design/dev_ddoc0/_view/views2?stale=false&connection_timeout=60000&full_set=true [2018-01-25 21:34:55,884] - [rest_client:551] INFO - index query url: http://cb-example-0003.cb-example.default.svc:8092/standard_bucket0/_design/dev_ddoc0/_view/views2?stale=false&connection_timeout=60000&full_set=true [2018-01-25 21:34:55,904] - [task:2384] INFO - Server: 172.17.0.8, Design Doc: dev_ddoc0, View: views2, (1000 rows) expected, (1000 rows) returned [2018-01-25 21:34:55,904] - [task:2391] INFO - expected number of rows: '1000' was found for view query [2018-01-25 21:34:55,904] - [createdeleteview:278] INFO - DDoc Data Validation Successful [2018-01-25 21:34:55,905] - [basetestcase:414] INFO - ============== basetestcase cleanup was started for test #15 test_view_ops ============== [2018-01-25 21:34:55,919] - [basetestcase:422] WARNING - Alerts were found: [{u'msg': u"IP address seems to have changed. Unable to listen on 'ns_1@cb-example-0003.cb-example.default.svc'. (POSIX error code: 'nxdomain')", u'serverTime': u'2018-01-25T21:28:43.000Z'}] [2018-01-25 21:34:55,956] - [bucket_helper:142] INFO - deleting existing buckets [u'bucket0', u'default', u'standard_bucket0'] on 172.17.0.8 [2018-01-25 21:34:55,956] - [bucket_helper:144] INFO - remove bucket bucket0 ... [2018-01-25 21:34:57,283] - [bucket_helper:158] INFO - deleted bucket : bucket0 from 172.17.0.8 [2018-01-25 21:34:57,284] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:34:57,305] - [rest_client:133] INFO - node 172.17.0.8 existing buckets : [u'default', u'standard_bucket0'] [2018-01-25 21:34:57,306] - [bucket_helper:144] INFO - remove bucket default ... [2018-01-25 21:34:58,866] - [bucket_helper:158] INFO - deleted bucket : default from 172.17.0.8 [2018-01-25 21:34:58,867] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:34:58,876] - [rest_client:133] INFO - node 172.17.0.8 existing buckets : [u'standard_bucket0'] [2018-01-25 21:34:58,877] - [bucket_helper:144] INFO - remove bucket standard_bucket0 ... [2018-01-25 21:35:00,581] - [bucket_helper:158] INFO - deleted bucket : standard_bucket0 from 172.17.0.8 [2018-01-25 21:35:00,582] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:35:00,585] - [rest_client:133] INFO - node 172.17.0.8 existing buckets : [] [2018-01-25 21:35:00,585] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:35:02,600] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.6 [2018-01-25 21:35:02,601] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:35:04,611] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.7 [2018-01-25 21:35:04,611] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:35:06,620] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.5 [2018-01-25 21:35:06,620] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:35:08,648] - [cluster_helper:78] INFO - waiting for ns_server @ 172.17.0.8:8091 [2018-01-25 21:35:08,654] - [cluster_helper:80] INFO - ns_server @ 172.17.0.8:8091 is running [2018-01-25 21:35:08,657] - [cluster_helper:78] INFO - waiting for ns_server @ 172.17.0.6:8091 [2018-01-25 21:35:08,661] - [cluster_helper:80] INFO - ns_server @ 172.17.0.6:8091 is running [2018-01-25 21:35:08,664] - [cluster_helper:78] INFO - waiting for ns_server @ 172.17.0.7:8091 [2018-01-25 21:35:08,667] - [cluster_helper:80] INFO - ns_server @ 172.17.0.7:8091 is running [2018-01-25 21:35:08,671] - [cluster_helper:78] INFO - waiting for ns_server @ 172.17.0.5:8091 [2018-01-25 21:35:08,674] - [cluster_helper:80] INFO - ns_server @ 172.17.0.5:8091 is running [2018-01-25 21:35:08,675] - [basetestcase:432] INFO - ============== basetestcase cleanup was finished for test #15 test_view_ops ============== test_view_ops (view.createdeleteview.CreateDeleteViewTests) ... ok ---------------------------------------------------------------------- Ran 1 test in 190.023s OK Cluster instance shutdown with force summary so far suite recreatebuckettests.RecreateMembaseBuckets , pass 0 , fail 1 failures so far... recreatebuckettests.RecreateMembaseBuckets.test_default_moxi summary so far suite deletebuckettests.DeleteMembaseBuckets , pass 1 , fail 0 summary so far suite createbuckettests.CreateMembaseBucketsTests , pass 1 , fail 4 failures so far... createbuckettests.CreateMembaseBucketsTests.test_default_moxi createbuckettests.CreateMembaseBucketsTests.test_default_on_non_default_port createbuckettests.CreateMembaseBucketsTests.test_non_default_case_sensitive_different_port createbuckettests.CreateMembaseBucketsTests.test_two_replica summary so far suite setgettests.MembaseBucket , pass 1 , fail 0 summary so far suite expirytests.ExpiryTests , pass 1 , fail 0 summary so far suite memcapable.GetlTests , pass 2 , fail 0 summary so far suite memorysanitytests.MemorySanity , pass 1 , fail 0 summary so far suite drainratetests.DrainRateTests , pass 1 , fail 0 summary so far suite view.viewquerytests.ViewQueryTests , pass 1 , fail 0 summary so far suite view.createdeleteview.CreateDeleteViewTests , pass 1 , fail 0 testrunner logs, diags and results are available under /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_15 Logs will be stored at /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_16 ./testrunner -i ./4node.ini -t rebalance.rebalancein.RebalanceInTests.rebalance_in_with_ops,nodes_in=3,replicas=1,items=1000,doc_ops=create;update;delete Test Input params: {'replicas': '1', 'doc_ops': 'create;update;delete', 'items': '1000', 'nodes_in': '3', 'conf_file': './4node.conf', 'num_nodes': 4, 'cluster_name': '4node', 'ini': './4node.ini', 'case_number': 16, 'logs_folder': '/testrunner/logs/testrunner-18-Jan-25_21-16-28/test_16', 'spec': '4node'} [2018-01-25 21:35:08,705] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 21:35:08,837] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 21:35:09,314] - [rest_client:1964] INFO - Node version in cluster 5.0.1-5003-enterprise [2018-01-25 21:35:09,341] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise'] [2018-01-25 21:35:09,341] - [basetestcase:201] INFO - ============== basetestcase setup was started for test #16 rebalance_in_with_ops============== [2018-01-25 21:35:09,365] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.8 [2018-01-25 21:35:09,365] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:35:11,374] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.6 [2018-01-25 21:35:11,374] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:35:13,381] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.7 [2018-01-25 21:35:13,381] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:35:15,390] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.5 [2018-01-25 21:35:15,391] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:35:17,411] - [basetestcase:482] INFO - sleep for 10 secs. ... [2018-01-25 21:35:27,430] - [cluster_helper:78] INFO - waiting for ns_server @ 172.17.0.8:8091 [2018-01-25 21:35:27,434] - [cluster_helper:80] INFO - ns_server @ 172.17.0.8:8091 is running [2018-01-25 21:35:27,437] - [cluster_helper:78] INFO - waiting for ns_server @ 172.17.0.6:8091 [2018-01-25 21:35:27,440] - [cluster_helper:80] INFO - ns_server @ 172.17.0.6:8091 is running [2018-01-25 21:35:27,443] - [cluster_helper:78] INFO - waiting for ns_server @ 172.17.0.7:8091 [2018-01-25 21:35:27,446] - [cluster_helper:80] INFO - ns_server @ 172.17.0.7:8091 is running [2018-01-25 21:35:27,451] - [cluster_helper:78] INFO - waiting for ns_server @ 172.17.0.5:8091 [2018-01-25 21:35:27,455] - [cluster_helper:80] INFO - ns_server @ 172.17.0.5:8091 is running [2018-01-25 21:35:27,455] - [basetestcase:245] INFO - initializing cluster [2018-01-25 21:35:27,740] - [task:125] INFO - server: ip:172.17.0.8 port:8091 ssh_username:root, nodes/self: {'ip': u'cb-example-0003.cb-example.default.svc', 'availableStorage': [], 'rest_username': '', 'id': u'ns_1@cb-example-0003.cb-example.default.svc', 'uptime': u'1372', 'mcdMemoryReserved': 6234, 'storageTotalRam': 7793, 'hostname': u'cb-example-0003.cb-example.default.svc:8091', 'storage': [], 'moxi': 11211, 'port': u'8091', 'version': u'5.0.1-5003-enterprise', 'memcached': 11210, 'status': u'healthy', 'clusterCompatibility': 327680, 'curr_items': 0, 'services': [u'fts', u'index', u'kv', u'n1ql'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 5594742784, 'memoryTotal': 8171810816, 'memoryQuota': 3900, 'mcdMemoryAllocated': 6234, 'os': u'x86_64-unknown-linux-gnu', 'ports': []} [2018-01-25 21:35:27,741] - [task:157] INFO - quota for index service will be 256 MB [2018-01-25 21:35:27,741] - [task:159] INFO - set index quota to node 172.17.0.8 [2018-01-25 21:35:27,741] - [rest_client:919] INFO - pools/default params : indexMemoryQuota=256 [2018-01-25 21:35:27,754] - [rest_client:907] INFO - pools/default params : memoryQuota=3900 [2018-01-25 21:35:27,815] - [rest_client:942] INFO - settings/indexes params : storageMode=plasma [2018-01-25 21:35:27,822] - [rest_client:807] ERROR - POST http://172.17.0.8:8091/settings/indexes body: storageMode=plasma headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 400 reason: unknown {"errors":{"storageMode":"Changing the optimization mode of global indexes is not supported when index service nodes are present in the cluster. Please remove all index service nodes to change this option."}} auth: Administrator:password [2018-01-25 21:35:27,823] - [rest_client:885] INFO - /node/controller/setupServices params on 172.17.0.8: 8091:services=n1ql%2Ckv%2Cindex&password=password&hostname=172.17.0.8&user=Administrator [2018-01-25 21:35:27,828] - [rest_client:807] ERROR - POST http://172.17.0.8:8091//node/controller/setupServices body: services=n1ql%2Ckv%2Cindex&password=password&hostname=172.17.0.8&user=Administrator headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 400 reason: unknown ["cannot change node services after cluster is provisioned"] auth: Administrator:password [2018-01-25 21:35:27,829] - [rest_client:890] INFO - This node is already provisioned with services, we do not consider this as failure for test case [2018-01-25 21:35:27,829] - [rest_client:825] INFO - settings/web params on 172.17.0.8:8091:username=Administrator&password=password&port=8091 [2018-01-25 21:35:27,951] - [task:125] INFO - server: ip:172.17.0.6 port:8091 ssh_username:root, nodes/self: {'ip': u'172.17.0.6', 'availableStorage': [], 'rest_username': '', 'id': u'ns_1@172.17.0.6', 'uptime': u'1398', 'mcdMemoryReserved': 6234, 'storageTotalRam': 7793, 'hostname': u'172.17.0.6:8091', 'storage': [], 'moxi': 11211, 'port': u'8091', 'version': u'5.0.1-5003-enterprise', 'memcached': 11210, 'status': u'healthy', 'clusterCompatibility': 327680, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 5587038208, 'memoryTotal': 8171810816, 'memoryQuota': 4156, 'mcdMemoryAllocated': 6234, 'os': u'x86_64-unknown-linux-gnu', 'ports': []} [2018-01-25 21:35:27,952] - [rest_client:907] INFO - pools/default params : memoryQuota=4156 [2018-01-25 21:35:27,959] - [rest_client:942] INFO - settings/indexes params : storageMode=plasma [2018-01-25 21:35:28,000] - [rest_client:825] INFO - settings/web params on 172.17.0.6:8091:username=Administrator&password=password&port=8091 [2018-01-25 21:35:28,076] - [task:125] INFO - server: ip:172.17.0.7 port:8091 ssh_username:root, nodes/self: {'ip': u'172.17.0.7', 'availableStorage': [], 'rest_username': '', 'id': u'ns_1@172.17.0.7', 'uptime': u'1387', 'mcdMemoryReserved': 6234, 'storageTotalRam': 7793, 'hostname': u'172.17.0.7:8091', 'storage': [], 'moxi': 11211, 'port': u'8091', 'version': u'5.0.1-5003-enterprise', 'memcached': 11210, 'status': u'healthy', 'clusterCompatibility': 327680, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 5595377664, 'memoryTotal': 8171810816, 'memoryQuota': 4156, 'mcdMemoryAllocated': 6234, 'os': u'x86_64-unknown-linux-gnu', 'ports': []} [2018-01-25 21:35:28,076] - [rest_client:907] INFO - pools/default params : memoryQuota=4156 [2018-01-25 21:35:28,080] - [rest_client:942] INFO - settings/indexes params : storageMode=plasma [2018-01-25 21:35:28,092] - [rest_client:825] INFO - settings/web params on 172.17.0.7:8091:username=Administrator&password=password&port=8091 [2018-01-25 21:35:28,172] - [task:125] INFO - server: ip:172.17.0.5 port:8091 ssh_username:root, nodes/self: {'ip': u'172.17.0.5', 'availableStorage': [], 'rest_username': '', 'id': u'ns_1@172.17.0.5', 'uptime': u'1425', 'mcdMemoryReserved': 6234, 'storageTotalRam': 7793, 'hostname': u'172.17.0.5:8091', 'storage': [], 'moxi': 11211, 'port': u'8091', 'version': u'5.0.1-5003-enterprise', 'memcached': 11210, 'status': u'healthy', 'clusterCompatibility': 327680, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 5594869760, 'memoryTotal': 8171810816, 'memoryQuota': 4156, 'mcdMemoryAllocated': 6234, 'os': u'x86_64-unknown-linux-gnu', 'ports': []} [2018-01-25 21:35:28,173] - [rest_client:907] INFO - pools/default params : memoryQuota=4156 [2018-01-25 21:35:28,178] - [rest_client:942] INFO - settings/indexes params : storageMode=plasma [2018-01-25 21:35:28,193] - [rest_client:825] INFO - settings/web params on 172.17.0.5:8091:username=Administrator&password=password&port=8091 [2018-01-25 21:35:28,275] - [basetestcase:2060] INFO - **** add built-in 'cbadminbucket' user to node 172.17.0.8 **** [2018-01-25 21:35:28,351] - [basetestcase:482] INFO - sleep for 10 secs. ... [2018-01-25 21:35:38,362] - [basetestcase:2065] INFO - **** add 'admin' role to 'cbadminbucket' user **** [2018-01-25 21:35:38,389] - [basetestcase:482] INFO - sleep for 10 secs. ... [2018-01-25 21:35:48,398] - [basetestcase:269] INFO - done initializing cluster [2018-01-25 21:35:49,320] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise'] [2018-01-25 21:35:49,321] - [rest_client:2237] INFO - http://172.17.0.8:8091/pools/default/buckets with param: bucketType=membase&threadsNumber=3&authType=sasl&replicaIndex=1&name=default&saslPassword=None&evictionPolicy=valueOnly&flushEnabled=1&replicaNumber=1&ramQuotaMB=3900 [2018-01-25 21:35:49,329] - [rest_client:2262] INFO - 0.01 seconds to create bucket default [2018-01-25 21:35:49,329] - [bucket_helper:343] INFO - waiting for memcached bucket : default in 172.17.0.8 to accept set ops [2018-01-25 21:35:50,658] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:35:50,742] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:35:52,352] - [task:312] INFO - bucket 'default' was created with per node RAM quota: 3900 [2018-01-25 21:35:52,988] - [basetestcase:344] INFO - ============== basetestcase setup was finished for test #16 rebalance_in_with_ops ============== [2018-01-25 21:35:53,006] - [basetestcase:482] INFO - sleep for 10 secs. ... [2018-01-25 21:36:03,051] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:36:04,053] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:36:04,071] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:36:04,079] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:36:04,087] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:36:06,011] - [task:425] INFO - adding node 172.17.0.6:8091 to cluster [2018-01-25 21:36:06,011] - [rest_client:1205] INFO - adding remote node @172.17.0.6:8091 to this cluster @172.17.0.8:8091 [2018-01-25 21:36:11,183] - [task:425] INFO - adding node 172.17.0.7:8091 to cluster [2018-01-25 21:36:11,183] - [rest_client:1205] INFO - adding remote node @172.17.0.7:8091 to this cluster @172.17.0.8:8091 [2018-01-25 21:36:16,255] - [task:425] INFO - adding node 172.17.0.5:8091 to cluster [2018-01-25 21:36:16,255] - [rest_client:1205] INFO - adding remote node @172.17.0.5:8091 to this cluster @172.17.0.8:8091 [2018-01-25 21:36:20,653] - [rest_client:1423] INFO - rebalance params : {'password': 'password', 'ejectedNodes': '', 'user': 'Administrator', 'knownNodes': u'ns_1@cb-example-0003.cb-example.default.svc,ns_1@172.17.0.7,ns_1@172.17.0.5,ns_1@172.17.0.6'} [2018-01-25 21:36:20,668] - [rest_client:1428] INFO - rebalance operation started [2018-01-25 21:36:20,695] - [rest_client:1576] INFO - rebalance percentage : 0.00 % [2018-01-25 21:36:20,697] - [task:491] INFO - Rebalance - status: running, progress: 0 [2018-01-25 21:36:30,993] - [rest_client:1576] INFO - rebalance percentage : 3.75 % [2018-01-25 21:36:31,035] - [task:491] INFO - Rebalance - status: running, progress: 3.75 [2018-01-25 21:36:41,631] - [rest_client:1576] INFO - rebalance percentage : 8.54 % [2018-01-25 21:36:41,631] - [task:491] INFO - Rebalance - status: running, progress: 8.54166666667 [2018-01-25 21:36:51,723] - [rest_client:1576] INFO - rebalance percentage : 14.48 % [2018-01-25 21:36:51,723] - [task:491] INFO - Rebalance - status: running, progress: 14.4791666667 [2018-01-25 21:37:01,800] - [rest_client:1576] INFO - rebalance percentage : 21.67 % [2018-01-25 21:37:01,800] - [task:491] INFO - Rebalance - status: running, progress: 21.6666666667 [2018-01-25 21:37:11,896] - [rest_client:1576] INFO - rebalance percentage : 26.67 % [2018-01-25 21:37:11,896] - [task:491] INFO - Rebalance - status: running, progress: 26.6666666667 [2018-01-25 21:37:21,974] - [rest_client:1576] INFO - rebalance percentage : 33.02 % [2018-01-25 21:37:21,974] - [task:491] INFO - Rebalance - status: running, progress: 33.0208333333 [2018-01-25 21:37:32,170] - [rest_client:1576] INFO - rebalance percentage : 36.56 % [2018-01-25 21:37:32,176] - [task:491] INFO - Rebalance - status: running, progress: 36.5625 [2018-01-25 21:37:43,062] - [rest_client:1576] INFO - rebalance percentage : 40.52 % [2018-01-25 21:37:43,107] - [task:491] INFO - Rebalance - status: running, progress: 40.5208333333 [2018-01-25 21:37:54,214] - [rest_client:1576] INFO - rebalance percentage : 42.81 % [2018-01-25 21:37:54,214] - [task:491] INFO - Rebalance - status: running, progress: 42.8125 [2018-01-25 21:38:04,397] - [rest_client:1576] INFO - rebalance percentage : 45.42 % [2018-01-25 21:38:04,397] - [task:491] INFO - Rebalance - status: running, progress: 45.4166666667 [2018-01-25 21:38:15,470] - [rest_client:1576] INFO - rebalance percentage : 49.38 % [2018-01-25 21:38:15,470] - [task:491] INFO - Rebalance - status: running, progress: 49.375 [2018-01-25 21:38:25,982] - [rest_client:1576] INFO - rebalance percentage : 55.73 % [2018-01-25 21:38:25,983] - [task:491] INFO - Rebalance - status: running, progress: 55.7291666667 [2018-01-25 21:38:36,233] - [rest_client:1576] INFO - rebalance percentage : 57.71 % [2018-01-25 21:38:36,233] - [task:491] INFO - Rebalance - status: running, progress: 57.7083333333 [2018-01-25 21:38:47,255] - [rest_client:1576] INFO - rebalance percentage : 62.81 % [2018-01-25 21:38:47,255] - [task:491] INFO - Rebalance - status: running, progress: 62.8125 [2018-01-25 21:38:58,178] - [rest_client:1576] INFO - rebalance percentage : 66.25 % [2018-01-25 21:38:58,178] - [task:491] INFO - Rebalance - status: running, progress: 66.25 [2018-01-25 21:39:08,461] - [rest_client:1576] INFO - rebalance percentage : 70.52 % [2018-01-25 21:39:08,462] - [task:491] INFO - Rebalance - status: running, progress: 70.5208333333 [2018-01-25 21:39:19,144] - [rest_client:1576] INFO - rebalance percentage : 72.60 % [2018-01-25 21:39:19,145] - [task:491] INFO - Rebalance - status: running, progress: 72.6041666667 [2018-01-25 21:39:29,481] - [rest_client:1576] INFO - rebalance percentage : 76.25 % [2018-01-25 21:39:29,483] - [task:491] INFO - Rebalance - status: running, progress: 76.25 [2018-01-25 21:39:39,721] - [rest_client:1576] INFO - rebalance percentage : 80.16 % [2018-01-25 21:39:39,723] - [task:491] INFO - Rebalance - status: running, progress: 80.15625 [2018-01-25 21:39:50,005] - [rest_client:1576] INFO - rebalance percentage : 81.24 % [2018-01-25 21:39:50,018] - [task:491] INFO - Rebalance - status: running, progress: 81.2369791667 [2018-01-25 21:40:00,463] - [rest_client:1576] INFO - rebalance percentage : 82.46 % [2018-01-25 21:40:00,463] - [task:491] INFO - Rebalance - status: running, progress: 82.4609375 [2018-01-25 21:40:10,976] - [rest_client:1576] INFO - rebalance percentage : 83.27 % [2018-01-25 21:40:10,977] - [task:491] INFO - Rebalance - status: running, progress: 83.2682291667 [2018-01-25 21:40:21,083] - [task:491] INFO - Rebalance - status: none, progress: 100 [2018-01-25 21:40:21,090] - [task:550] INFO - rebalancing was completed with progress: 100% in 240.42095089 sec [2018-01-25 21:40:21,100] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:40:21,107] - [basetestcase:2242] INFO - list of all nodes in cluster: [ip:172.17.0.7 port:8091 ssh_username:root, ip:172.17.0.5 port:8091 ssh_username:root, ip:172.17.0.6 port:8091 ssh_username:root] [2018-01-25 21:40:21,114] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:40:21,121] - [basetestcase:2242] INFO - list of all nodes in cluster: [ip:172.17.0.7 port:8091 ssh_username:root, ip:172.17.0.5 port:8091 ssh_username:root, ip:172.17.0.6 port:8091 ssh_username:root] [2018-01-25 21:40:22,101] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 default [2018-01-25 21:40:22,156] - [task:602] INFO - Saw ep_queue_size 0 == 0 expected on '172.17.0.6:8091',default bucket [2018-01-25 21:40:22,165] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 default [2018-01-25 21:40:22,223] - [task:602] INFO - Saw ep_queue_size 0 == 0 expected on '172.17.0.7:8091',default bucket [2018-01-25 21:40:22,234] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 default [2018-01-25 21:40:22,287] - [task:602] INFO - Saw ep_queue_size 0 == 0 expected on '172.17.0.5:8091',default bucket [2018-01-25 21:40:22,335] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 default [2018-01-25 21:40:22,393] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 default [2018-01-25 21:40:22,475] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 default [2018-01-25 21:40:22,526] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:40:22,606] - [task:1756] INFO - 1000 items will be verified on default bucket [2018-01-25 21:40:23,290] - [task:1765] INFO - 0 items were verified [2018-01-25 21:40:23,385] - [task:1769] INFO - 1000 items were verified in 0.778878927231 sec.the average number of ops - 1283.89344763 per second [2018-01-25 21:40:23,400] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:40:23,407] - [basetestcase:2242] INFO - list of all nodes in cluster: [ip:172.17.0.7 port:8091 ssh_username:root, ip:172.17.0.5 port:8091 ssh_username:root, ip:172.17.0.6 port:8091 ssh_username:root] [2018-01-25 21:40:24,306] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 default [2018-01-25 21:40:24,358] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 default [2018-01-25 21:40:24,453] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 default [2018-01-25 21:40:24,502] - [task:598] WARNING - Not Ready: curr_items 751 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:40:24,510] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 default [2018-01-25 21:40:24,560] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 default [2018-01-25 21:40:24,626] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 default [2018-01-25 21:40:24,688] - [task:598] WARNING - Not Ready: vb_active_curr_items 751 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:40:24,701] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 default [2018-01-25 21:40:24,773] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 default [2018-01-25 21:40:24,832] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 default [2018-01-25 21:40:24,876] - [task:598] WARNING - Not Ready: vb_replica_curr_items 752 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:40:24,884] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 default [2018-01-25 21:40:24,942] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 default [2018-01-25 21:40:25,017] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 default [2018-01-25 21:40:25,104] - [task:598] WARNING - Not Ready: curr_items_tot 1503 == 2000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:40:30,122] - [task:598] WARNING - Not Ready: curr_items 751 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:40:30,132] - [task:598] WARNING - Not Ready: vb_active_curr_items 751 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:40:30,147] - [task:598] WARNING - Not Ready: vb_replica_curr_items 752 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:40:30,157] - [task:598] WARNING - Not Ready: curr_items_tot 1503 == 2000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:40:35,171] - [task:598] WARNING - Not Ready: curr_items 751 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:40:35,181] - [task:598] WARNING - Not Ready: vb_active_curr_items 751 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:40:35,191] - [task:598] WARNING - Not Ready: vb_replica_curr_items 752 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:40:35,201] - [task:598] WARNING - Not Ready: curr_items_tot 1503 == 2000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:40:40,219] - [task:598] WARNING - Not Ready: curr_items 751 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:40:40,230] - [task:598] WARNING - Not Ready: vb_active_curr_items 751 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:40:40,241] - [task:598] WARNING - Not Ready: vb_replica_curr_items 752 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:40:40,252] - [task:598] WARNING - Not Ready: curr_items_tot 1503 == 2000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:40:45,273] - [task:598] WARNING - Not Ready: curr_items 751 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:40:45,284] - [task:598] WARNING - Not Ready: vb_active_curr_items 751 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:40:45,297] - [task:598] WARNING - Not Ready: vb_replica_curr_items 752 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:40:45,309] - [task:598] WARNING - Not Ready: curr_items_tot 1503 == 2000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:40:50,323] - [task:598] WARNING - Not Ready: curr_items 751 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:40:50,334] - [task:598] WARNING - Not Ready: vb_active_curr_items 751 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:40:50,343] - [task:598] WARNING - Not Ready: vb_replica_curr_items 752 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:40:50,353] - [task:598] WARNING - Not Ready: curr_items_tot 1503 == 2000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:40:55,368] - [task:598] WARNING - Not Ready: curr_items 751 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:40:55,377] - [task:598] WARNING - Not Ready: vb_active_curr_items 751 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:40:55,387] - [task:598] WARNING - Not Ready: vb_replica_curr_items 752 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:40:55,397] - [task:598] WARNING - Not Ready: curr_items_tot 1503 == 2000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:41:00,415] - [task:598] WARNING - Not Ready: curr_items 751 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:41:00,438] - [task:598] WARNING - Not Ready: vb_active_curr_items 751 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:41:00,449] - [task:598] WARNING - Not Ready: vb_replica_curr_items 752 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:41:00,461] - [task:598] WARNING - Not Ready: curr_items_tot 1503 == 2000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:41:05,477] - [task:598] WARNING - Not Ready: curr_items 751 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:41:05,489] - [task:598] WARNING - Not Ready: vb_active_curr_items 751 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:41:05,500] - [task:598] WARNING - Not Ready: vb_replica_curr_items 752 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:41:05,511] - [task:598] WARNING - Not Ready: curr_items_tot 1503 == 2000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:41:10,535] - [task:598] WARNING - Not Ready: curr_items 751 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:41:10,554] - [task:598] WARNING - Not Ready: vb_active_curr_items 751 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:41:10,569] - [task:598] WARNING - Not Ready: vb_replica_curr_items 752 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:41:10,591] - [task:598] WARNING - Not Ready: curr_items_tot 1503 == 2000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:41:15,609] - [task:598] WARNING - Not Ready: curr_items 751 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:41:15,620] - [task:598] WARNING - Not Ready: vb_active_curr_items 751 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:41:15,631] - [task:598] WARNING - Not Ready: vb_replica_curr_items 752 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:41:15,641] - [task:598] WARNING - Not Ready: curr_items_tot 1503 == 2000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:41:20,659] - [task:598] WARNING - Not Ready: curr_items 751 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:41:20,673] - [task:598] WARNING - Not Ready: vb_active_curr_items 751 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:41:20,687] - [task:598] WARNING - Not Ready: vb_replica_curr_items 752 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:41:20,700] - [task:598] WARNING - Not Ready: curr_items_tot 1503 == 2000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:41:25,745] - [task:598] WARNING - Not Ready: curr_items 751 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:41:25,770] - [task:598] WARNING - Not Ready: vb_active_curr_items 751 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:41:25,795] - [task:598] WARNING - Not Ready: vb_replica_curr_items 752 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:41:25,816] - [task:598] WARNING - Not Ready: curr_items_tot 1503 == 2000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:41:30,871] - [task:598] WARNING - Not Ready: curr_items 751 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:41:30,915] - [task:598] WARNING - Not Ready: vb_active_curr_items 751 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:41:30,955] - [task:598] WARNING - Not Ready: vb_replica_curr_items 752 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:41:30,989] - [task:598] WARNING - Not Ready: curr_items_tot 1503 == 2000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:41:36,008] - [task:598] WARNING - Not Ready: curr_items 751 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:41:36,020] - [task:598] WARNING - Not Ready: vb_active_curr_items 751 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:41:36,030] - [task:598] WARNING - Not Ready: vb_replica_curr_items 752 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:41:36,040] - [task:598] WARNING - Not Ready: curr_items_tot 1503 == 2000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:41:41,061] - [task:598] WARNING - Not Ready: curr_items 751 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:41:41,073] - [task:598] WARNING - Not Ready: vb_active_curr_items 751 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:41:41,086] - [task:598] WARNING - Not Ready: vb_replica_curr_items 752 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:41:41,103] - [task:598] WARNING - Not Ready: curr_items_tot 1503 == 2000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:41:46,124] - [task:598] WARNING - Not Ready: curr_items 751 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:41:46,138] - [task:598] WARNING - Not Ready: vb_active_curr_items 751 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:41:46,154] - [task:598] WARNING - Not Ready: vb_replica_curr_items 752 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:41:46,174] - [task:598] WARNING - Not Ready: curr_items_tot 1503 == 2000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:41:51,192] - [task:598] WARNING - Not Ready: curr_items 751 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:41:51,204] - [task:598] WARNING - Not Ready: vb_active_curr_items 751 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:41:51,229] - [task:598] WARNING - Not Ready: vb_replica_curr_items 752 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:41:51,239] - [task:598] WARNING - Not Ready: curr_items_tot 1503 == 2000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:41:56,269] - [task:598] WARNING - Not Ready: curr_items 751 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:41:56,293] - [task:598] WARNING - Not Ready: vb_active_curr_items 751 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:41:56,341] - [task:598] WARNING - Not Ready: vb_replica_curr_items 752 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:41:56,368] - [task:598] WARNING - Not Ready: curr_items_tot 1503 == 2000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:42:01,389] - [task:598] WARNING - Not Ready: curr_items 751 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:42:01,400] - [task:598] WARNING - Not Ready: vb_active_curr_items 751 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:42:01,410] - [task:598] WARNING - Not Ready: vb_replica_curr_items 752 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:42:01,420] - [task:598] WARNING - Not Ready: curr_items_tot 1503 == 2000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:42:06,439] - [task:598] WARNING - Not Ready: curr_items 751 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:42:06,451] - [task:598] WARNING - Not Ready: vb_active_curr_items 751 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:42:06,474] - [task:598] WARNING - Not Ready: vb_replica_curr_items 752 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:42:06,485] - [task:598] WARNING - Not Ready: curr_items_tot 1503 == 2000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:42:11,503] - [task:598] WARNING - Not Ready: curr_items 751 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:42:11,542] - [task:598] WARNING - Not Ready: vb_active_curr_items 751 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:42:11,556] - [task:598] WARNING - Not Ready: vb_replica_curr_items 752 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:42:11,579] - [task:598] WARNING - Not Ready: curr_items_tot 1503 == 2000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:42:16,595] - [task:598] WARNING - Not Ready: curr_items 751 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:42:16,606] - [task:598] WARNING - Not Ready: vb_active_curr_items 751 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:42:16,617] - [task:598] WARNING - Not Ready: vb_replica_curr_items 752 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:42:16,628] - [task:598] WARNING - Not Ready: curr_items_tot 1503 == 2000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:42:21,651] - [task:598] WARNING - Not Ready: curr_items 751 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:42:21,663] - [task:598] WARNING - Not Ready: vb_active_curr_items 751 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:42:21,675] - [task:598] WARNING - Not Ready: vb_replica_curr_items 752 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:42:21,688] - [task:598] WARNING - Not Ready: curr_items_tot 1503 == 2000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [('/usr/lib/python2.7/threading.py', 783, '__bootstrap', 'self.__bootstrap_inner()'), ('/usr/lib/python2.7/threading.py', 810, '__bootstrap_inner', 'self.run()'), ('./testrunner.py', 297, 'run', '**self._Thread__kwargs)'), ('/usr/lib/python2.7/unittest/runner.py', 151, 'run', 'test(result)'), ('/usr/lib/python2.7/unittest/case.py', 395, '__call__', 'return self.run(*args, **kwds)'), ('/usr/lib/python2.7/unittest/case.py', 331, 'run', 'testMethod()'), ('pytests/rebalance/rebalancein.py', 171, 'rebalance_in_with_ops', 'self.verify_cluster_stats(self.servers[:self.nodes_in + self.nodes_init])'), ('pytests/basetestcase.py', 1143, 'verify_cluster_stats', 'self._verify_stats_all_buckets(servers, timeout=(timeout or 120))'), ('pytests/basetestcase.py', 815, '_verify_stats_all_buckets', 'task.result(timeout)'), ('lib/tasks/future.py', 162, 'result', 'self.set_exception(TimeoutError())'), ('lib/tasks/future.py', 264, 'set_exception', 'print traceback.extract_stack()')] Thu Jan 25 21:42:23 2018 [2018-01-25 21:42:23,409] - [basetestcase:817] INFO - [2018-01-25 21:42:23,409] - [basetestcase:820] ERROR - unable to get expected stats for any node! Print taps for all nodes: [2018-01-25 21:42:23,459] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 default [2018-01-25 21:42:23,525] - [rebalance_helper:317] INFO - getting tap stats... for 172.17.0.5 [2018-01-25 21:42:23,525] - [rebalance_helper:323] ERROR - error Memcached error #1 'Not found': for vbucket :0 to mc 172.17.0.5:11210 while getting stats... [2018-01-25 21:42:23,535] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 default [2018-01-25 21:42:23,605] - [rebalance_helper:317] INFO - getting tap stats... for 172.17.0.6 [2018-01-25 21:42:23,606] - [rebalance_helper:323] ERROR - error Memcached error #1 'Not found': for vbucket :0 to mc 172.17.0.6:11210 while getting stats... [2018-01-25 21:42:23,619] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 default [2018-01-25 21:42:23,683] - [rebalance_helper:317] INFO - getting tap stats... for 172.17.0.7 [2018-01-25 21:42:23,684] - [rebalance_helper:323] ERROR - error Memcached error #1 'Not found': for vbucket :0 to mc 172.17.0.7:11210 while getting stats... [2018-01-25 21:42:23,699] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:42:23,777] - [rebalance_helper:317] INFO - getting tap stats... for cb-example-0003.cb-example.default.svc [2018-01-25 21:42:23,777] - [rebalance_helper:323] ERROR - error Memcached error #1 'Not found': for vbucket :0 to mc cb-example-0003.cb-example.default.svc:11210 while getting stats... rebalance_in_with_ops (rebalance.rebalancein.RebalanceInTests) ... ERROR [2018-01-25 21:42:23,780] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 21:42:23,912] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 21:42:24,264] - [remote_util:2995] INFO - running command.raw on 172.17.0.8: kill -9 $(ps aux | pgrep 'memcached') [2018-01-25 21:42:24,337] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:42:24,339] - [remote_util:232] INFO - connecting to 172.17.0.6 with username:root password:couchbase ssh_key: [2018-01-25 21:42:24,422] - [remote_util:266] INFO - Connected to 172.17.0.6 [2018-01-25 21:42:24,800] - [remote_util:2995] INFO - running command.raw on 172.17.0.6: kill -9 $(ps aux | pgrep 'memcached') [2018-01-25 21:42:24,876] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:42:24,991] - [remote_util:232] INFO - connecting to 172.17.0.7 with username:root password:couchbase ssh_key: [2018-01-25 21:42:25,122] - [remote_util:266] INFO - Connected to 172.17.0.7 [2018-01-25 21:42:25,560] - [remote_util:2995] INFO - running command.raw on 172.17.0.7: kill -9 $(ps aux | pgrep 'memcached') [2018-01-25 21:42:25,642] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:42:25,760] - [remote_util:232] INFO - connecting to 172.17.0.5 with username:root password:couchbase ssh_key: [2018-01-25 21:42:25,846] - [remote_util:266] INFO - Connected to 172.17.0.5 [2018-01-25 21:42:26,751] - [remote_util:2995] INFO - running command.raw on 172.17.0.5: kill -9 $(ps aux | pgrep 'memcached') [2018-01-25 21:42:26,849] - [remote_util:3032] INFO - command executed successfully ====================================================================== ERROR: rebalance_in_with_ops (rebalance.rebalancein.RebalanceInTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "pytests/rebalance/rebalancein.py", line 171, in rebalance_in_with_ops self.verify_cluster_stats(self.servers[:self.nodes_in + self.nodes_init]) File "pytests/basetestcase.py", line 1143, in verify_cluster_stats self._verify_stats_all_buckets(servers, timeout=(timeout or 120)) File "pytests/basetestcase.py", line 824, in _verify_stats_all_buckets raise Exception("unable to get expected stats during {0} sec".format(timeout)) Exception: unable to get expected stats during 120 sec ---------------------------------------------------------------------- Ran 1 test in 438.166s FAILED (errors=1) Cluster instance shutdown with force summary so far suite recreatebuckettests.RecreateMembaseBuckets , pass 0 , fail 1 failures so far... recreatebuckettests.RecreateMembaseBuckets.test_default_moxi summary so far suite deletebuckettests.DeleteMembaseBuckets , pass 1 , fail 0 summary so far suite createbuckettests.CreateMembaseBucketsTests , pass 1 , fail 4 failures so far... createbuckettests.CreateMembaseBucketsTests.test_default_moxi createbuckettests.CreateMembaseBucketsTests.test_default_on_non_default_port createbuckettests.CreateMembaseBucketsTests.test_non_default_case_sensitive_different_port createbuckettests.CreateMembaseBucketsTests.test_two_replica summary so far suite setgettests.MembaseBucket , pass 1 , fail 0 summary so far suite expirytests.ExpiryTests , pass 1 , fail 0 summary so far suite memcapable.GetlTests , pass 2 , fail 0 summary so far suite memorysanitytests.MemorySanity , pass 1 , fail 0 summary so far suite drainratetests.DrainRateTests , pass 1 , fail 0 summary so far suite view.viewquerytests.ViewQueryTests , pass 1 , fail 0 summary so far suite view.createdeleteview.CreateDeleteViewTests , pass 1 , fail 0 summary so far suite rebalance.rebalancein.RebalanceInTests , pass 0 , fail 1 failures so far... rebalance.rebalancein.RebalanceInTests.rebalance_in_with_ops testrunner logs, diags and results are available under /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_16 Logs will be stored at /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_17 ./testrunner -i ./4node.ini -t rebalance.rebalanceout.RebalanceOutTests.rebalance_out_with_ops,nodes_out=3,replicas=1,items=1000 Test Input params: {'replicas': '1', 'items': '1000', 'case_number': 17, 'conf_file': './4node.conf', 'num_nodes': 4, 'cluster_name': '4node', 'ini': './4node.ini', 'nodes_out': '3', 'logs_folder': '/testrunner/logs/testrunner-18-Jan-25_21-16-28/test_17', 'spec': '4node'} [2018-01-25 21:42:26,908] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 21:42:27,040] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 21:42:27,623] - [rest_client:47] WARNING - server 172.17.0.8:8091 status is warmup [2018-01-25 21:42:29,635] - [rest_client:47] WARNING - server 172.17.0.8:8091 status is warmup [2018-01-25 21:42:31,640] - [rest_client:47] WARNING - server 172.17.0.8:8091 status is warmup [2018-01-25 21:42:33,647] - [rest_client:47] WARNING - server 172.17.0.8:8091 status is warmup [2018-01-25 21:42:35,664] - [rest_client:1964] INFO - Node version in cluster 5.0.1-5003-enterprise [2018-01-25 21:42:35,695] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:42:35,696] - [basetestcase:201] INFO - ============== basetestcase setup was started for test #17 rebalance_out_with_ops============== [2018-01-25 21:42:35,745] - [bucket_helper:142] INFO - deleting existing buckets [u'default'] on 172.17.0.8 [2018-01-25 21:42:35,746] - [bucket_helper:144] INFO - remove bucket default ... [2018-01-25 21:42:37,654] - [bucket_helper:158] INFO - deleted bucket : default from 172.17.0.8 [2018-01-25 21:42:37,655] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:42:37,658] - [rest_client:133] INFO - node 172.17.0.8 existing buckets : [] [2018-01-25 21:42:37,658] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:42:39,668] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.6 [2018-01-25 21:42:39,668] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:42:41,679] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.7 [2018-01-25 21:42:41,679] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:42:43,693] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.5 [2018-01-25 21:42:43,693] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:42:45,722] - [cluster_helper:254] INFO - rebalancing all nodes in order to remove nodes [2018-01-25 21:42:45,724] - [rest_client:1423] INFO - rebalance params : {'password': 'password', 'ejectedNodes': u'ns_1@172.17.0.7,ns_1@172.17.0.5,ns_1@172.17.0.6', 'user': 'Administrator', 'knownNodes': u'ns_1@cb-example-0003.cb-example.default.svc,ns_1@172.17.0.7,ns_1@172.17.0.5,ns_1@172.17.0.6'} [2018-01-25 21:42:45,731] - [rest_client:1428] INFO - rebalance operation started [2018-01-25 21:42:45,751] - [rest_client:1576] INFO - rebalance percentage : 0.00 % [2018-01-25 21:43:05,761] - [rest_client:1515] INFO - rebalance progress took 20.0304591656 seconds [2018-01-25 21:43:05,761] - [rest_client:1516] INFO - sleep for 10 seconds after rebalance... [2018-01-25 21:43:15,792] - [cluster_helper:294] INFO - removed all the nodes from cluster associated with ip:172.17.0.8 port:8091 ssh_username:root ? [(u'ns_1@172.17.0.7', 8091), (u'ns_1@172.17.0.5', 8091), (u'ns_1@172.17.0.6', 8091)] [2018-01-25 21:43:15,792] - [basetestcase:482] INFO - sleep for 10 secs. ... [2018-01-25 21:43:25,804] - [cluster_helper:78] INFO - waiting for ns_server @ 172.17.0.8:8091 [2018-01-25 21:43:25,813] - [cluster_helper:80] INFO - ns_server @ 172.17.0.8:8091 is running [2018-01-25 21:43:25,817] - [cluster_helper:78] INFO - waiting for ns_server @ 172.17.0.6:8091 [2018-01-25 21:43:25,820] - [cluster_helper:80] INFO - ns_server @ 172.17.0.6:8091 is running [2018-01-25 21:43:25,826] - [cluster_helper:78] INFO - waiting for ns_server @ 172.17.0.7:8091 [2018-01-25 21:43:25,835] - [cluster_helper:80] INFO - ns_server @ 172.17.0.7:8091 is running [2018-01-25 21:43:25,839] - [cluster_helper:78] INFO - waiting for ns_server @ 172.17.0.5:8091 [2018-01-25 21:43:25,846] - [cluster_helper:80] INFO - ns_server @ 172.17.0.5:8091 is running [2018-01-25 21:43:25,846] - [basetestcase:245] INFO - initializing cluster [2018-01-25 21:43:26,002] - [task:125] INFO - server: ip:172.17.0.8 port:8091 ssh_username:root, nodes/self: {'ip': u'cb-example-0003.cb-example.default.svc', 'availableStorage': [], 'rest_username': '', 'id': u'ns_1@cb-example-0003.cb-example.default.svc', 'uptime': u'1852', 'mcdMemoryReserved': 6234, 'storageTotalRam': 7793, 'hostname': u'cb-example-0003.cb-example.default.svc:8091', 'storage': [], 'moxi': 11211, 'port': u'8091', 'version': u'5.0.1-5003-enterprise', 'memcached': 11210, 'status': u'healthy', 'clusterCompatibility': 327680, 'curr_items': 0, 'services': [u'fts', u'index', u'kv', u'n1ql'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 5693198336, 'memoryTotal': 8171810816, 'memoryQuota': 3900, 'mcdMemoryAllocated': 6234, 'os': u'x86_64-unknown-linux-gnu', 'ports': []} [2018-01-25 21:43:26,003] - [task:157] INFO - quota for index service will be 256 MB [2018-01-25 21:43:26,003] - [task:159] INFO - set index quota to node 172.17.0.8 [2018-01-25 21:43:26,003] - [rest_client:919] INFO - pools/default params : indexMemoryQuota=256 [2018-01-25 21:43:26,018] - [rest_client:907] INFO - pools/default params : memoryQuota=3900 [2018-01-25 21:43:26,071] - [rest_client:942] INFO - settings/indexes params : storageMode=plasma [2018-01-25 21:43:26,087] - [rest_client:807] ERROR - POST http://172.17.0.8:8091/settings/indexes body: storageMode=plasma headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 400 reason: unknown {"errors":{"storageMode":"Changing the optimization mode of global indexes is not supported when index service nodes are present in the cluster. Please remove all index service nodes to change this option."}} auth: Administrator:password [2018-01-25 21:43:26,088] - [rest_client:885] INFO - /node/controller/setupServices params on 172.17.0.8: 8091:services=n1ql%2Ckv%2Cindex&password=password&hostname=172.17.0.8&user=Administrator [2018-01-25 21:43:26,144] - [rest_client:807] ERROR - POST http://172.17.0.8:8091//node/controller/setupServices body: services=n1ql%2Ckv%2Cindex&password=password&hostname=172.17.0.8&user=Administrator headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 400 reason: unknown ["cannot change node services after cluster is provisioned"] auth: Administrator:password [2018-01-25 21:43:26,145] - [rest_client:890] INFO - This node is already provisioned with services, we do not consider this as failure for test case [2018-01-25 21:43:26,145] - [rest_client:825] INFO - settings/web params on 172.17.0.8:8091:username=Administrator&password=password&port=8091 [2018-01-25 21:43:26,556] - [task:125] INFO - server: ip:172.17.0.6 port:8091 ssh_username:root, nodes/self: {'ip': u'172.17.0.6', 'availableStorage': [], 'rest_username': '', 'id': u'ns_1@172.17.0.6', 'uptime': u'1878', 'mcdMemoryReserved': 6234, 'storageTotalRam': 7793, 'hostname': u'172.17.0.6:8091', 'storage': [], 'moxi': 11211, 'port': u'8091', 'version': u'5.0.1-5003-enterprise', 'memcached': 11210, 'status': u'healthy', 'clusterCompatibility': 327680, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 5684977664, 'memoryTotal': 8171810816, 'memoryQuota': 4675, 'mcdMemoryAllocated': 6234, 'os': u'x86_64-unknown-linux-gnu', 'ports': []} [2018-01-25 21:43:26,556] - [rest_client:907] INFO - pools/default params : memoryQuota=4156 [2018-01-25 21:43:26,569] - [rest_client:942] INFO - settings/indexes params : storageMode=plasma [2018-01-25 21:43:26,585] - [rest_client:825] INFO - settings/web params on 172.17.0.6:8091:username=Administrator&password=password&port=8091 [2018-01-25 21:43:26,908] - [task:125] INFO - server: ip:172.17.0.7 port:8091 ssh_username:root, nodes/self: {'ip': u'172.17.0.7', 'availableStorage': [], 'rest_username': '', 'id': u'ns_1@172.17.0.7', 'uptime': u'1868', 'mcdMemoryReserved': 6234, 'storageTotalRam': 7793, 'hostname': u'172.17.0.7:8091', 'storage': [], 'moxi': 11211, 'port': u'8091', 'version': u'5.0.1-5003-enterprise', 'memcached': 11210, 'status': u'healthy', 'clusterCompatibility': 327680, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 5692690432, 'memoryTotal': 8171810816, 'memoryQuota': 4675, 'mcdMemoryAllocated': 6234, 'os': u'x86_64-unknown-linux-gnu', 'ports': []} [2018-01-25 21:43:26,909] - [rest_client:907] INFO - pools/default params : memoryQuota=4156 [2018-01-25 21:43:26,918] - [rest_client:942] INFO - settings/indexes params : storageMode=plasma [2018-01-25 21:43:26,930] - [rest_client:825] INFO - settings/web params on 172.17.0.7:8091:username=Administrator&password=password&port=8091 [2018-01-25 21:43:27,032] - [task:125] INFO - server: ip:172.17.0.5 port:8091 ssh_username:root, nodes/self: {'ip': u'172.17.0.5', 'availableStorage': [], 'rest_username': '', 'id': u'ns_1@172.17.0.5', 'uptime': u'1903', 'mcdMemoryReserved': 6234, 'storageTotalRam': 7793, 'hostname': u'172.17.0.5:8091', 'storage': [], 'moxi': 11211, 'port': u'8091', 'version': u'5.0.1-5003-enterprise', 'memcached': 11210, 'status': u'healthy', 'clusterCompatibility': 327680, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 5693317120, 'memoryTotal': 8171810816, 'memoryQuota': 4675, 'mcdMemoryAllocated': 6234, 'os': u'x86_64-unknown-linux-gnu', 'ports': []} [2018-01-25 21:43:27,033] - [rest_client:907] INFO - pools/default params : memoryQuota=4156 [2018-01-25 21:43:27,040] - [rest_client:942] INFO - settings/indexes params : storageMode=plasma [2018-01-25 21:43:27,046] - [rest_client:825] INFO - settings/web params on 172.17.0.5:8091:username=Administrator&password=password&port=8091 [2018-01-25 21:43:27,242] - [basetestcase:2060] INFO - **** add built-in 'cbadminbucket' user to node 172.17.0.8 **** [2018-01-25 21:43:27,403] - [basetestcase:482] INFO - sleep for 10 secs. ... [2018-01-25 21:43:37,413] - [basetestcase:2065] INFO - **** add 'admin' role to 'cbadminbucket' user **** [2018-01-25 21:43:37,487] - [basetestcase:482] INFO - sleep for 10 secs. ... [2018-01-25 21:43:47,499] - [basetestcase:269] INFO - done initializing cluster [2018-01-25 21:43:48,238] - [task:425] INFO - adding node 172.17.0.6:8091 to cluster [2018-01-25 21:43:48,238] - [rest_client:1205] INFO - adding remote node @172.17.0.6:8091 to this cluster @172.17.0.8:8091 [2018-01-25 21:43:51,440] - [task:425] INFO - adding node 172.17.0.7:8091 to cluster [2018-01-25 21:43:51,441] - [rest_client:1205] INFO - adding remote node @172.17.0.7:8091 to this cluster @172.17.0.8:8091 [2018-01-25 21:43:55,683] - [task:425] INFO - adding node 172.17.0.5:8091 to cluster [2018-01-25 21:43:55,684] - [rest_client:1205] INFO - adding remote node @172.17.0.5:8091 to this cluster @172.17.0.8:8091 [2018-01-25 21:43:59,003] - [rest_client:1423] INFO - rebalance params : {'password': 'password', 'ejectedNodes': '', 'user': 'Administrator', 'knownNodes': u'ns_1@cb-example-0003.cb-example.default.svc,ns_1@172.17.0.7,ns_1@172.17.0.5,ns_1@172.17.0.6'} [2018-01-25 21:43:59,015] - [rest_client:1428] INFO - rebalance operation started [2018-01-25 21:43:59,034] - [rest_client:1576] INFO - rebalance percentage : 0.00 % [2018-01-25 21:43:59,036] - [task:491] INFO - Rebalance - status: running, progress: 0 [2018-01-25 21:44:09,145] - [task:491] INFO - Rebalance - status: none, progress: 100 [2018-01-25 21:44:09,154] - [task:550] INFO - rebalancing was completed with progress: 100% in 10.1383841038 sec [2018-01-25 21:44:10,170] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:44:10,171] - [rest_client:2237] INFO - http://172.17.0.8:8091/pools/default/buckets with param: bucketType=membase&threadsNumber=3&authType=sasl&replicaIndex=1&name=default&saslPassword=None&evictionPolicy=valueOnly&flushEnabled=1&replicaNumber=1&ramQuotaMB=3900 [2018-01-25 21:44:10,178] - [rest_client:2262] INFO - 0.01 seconds to create bucket default [2018-01-25 21:44:10,179] - [bucket_helper:343] INFO - waiting for memcached bucket : default in 172.17.0.8 to accept set ops [2018-01-25 21:44:12,342] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 default [2018-01-25 21:44:13,214] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 default [2018-01-25 21:44:13,973] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 default [2018-01-25 21:44:14,340] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:44:14,667] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 default [2018-01-25 21:44:15,029] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 default [2018-01-25 21:44:15,481] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 default [2018-01-25 21:44:15,539] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:44:15,819] - [task:312] INFO - bucket 'default' was created with per node RAM quota: 3900 [2018-01-25 21:44:15,831] - [basetestcase:344] INFO - ============== basetestcase setup was finished for test #17 rebalance_out_with_ops ============== [2018-01-25 21:44:15,839] - [basetestcase:482] INFO - sleep for 10 secs. ... [2018-01-25 21:44:25,890] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 default [2018-01-25 21:44:25,951] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 default [2018-01-25 21:44:26,005] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 default [2018-01-25 21:44:26,060] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:44:26,928] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 default [2018-01-25 21:44:26,934] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 default [2018-01-25 21:44:26,948] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 default [2018-01-25 21:44:26,959] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 default [2018-01-25 21:44:27,012] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 default [2018-01-25 21:44:27,018] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 default [2018-01-25 21:44:27,022] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 default [2018-01-25 21:44:27,049] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 default [2018-01-25 21:44:27,102] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 default [2018-01-25 21:44:27,106] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 default [2018-01-25 21:44:27,110] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 default [2018-01-25 21:44:27,177] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 default [2018-01-25 21:44:27,197] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:44:27,199] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:44:27,228] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:44:27,295] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:44:28,123] - [rest_client:1423] INFO - rebalance params : {'password': 'password', 'ejectedNodes': u'ns_1@172.17.0.5,ns_1@172.17.0.7,ns_1@172.17.0.6', 'user': 'Administrator', 'knownNodes': u'ns_1@cb-example-0003.cb-example.default.svc,ns_1@172.17.0.7,ns_1@172.17.0.5,ns_1@172.17.0.6'} [2018-01-25 21:44:28,220] - [rest_client:1428] INFO - rebalance operation started [2018-01-25 21:44:28,269] - [rest_client:1576] INFO - rebalance percentage : 0.00 % [2018-01-25 21:44:28,269] - [task:491] INFO - Rebalance - status: running, progress: 0 [2018-01-25 21:44:38,618] - [rest_client:1576] INFO - rebalance percentage : 4.95 % [2018-01-25 21:44:38,619] - [task:491] INFO - Rebalance - status: running, progress: 4.94791666667 [2018-01-25 21:44:49,025] - [rest_client:1576] INFO - rebalance percentage : 10.58 % [2018-01-25 21:44:49,038] - [task:491] INFO - Rebalance - status: running, progress: 10.576171875 [2018-01-25 21:44:59,171] - [rest_client:1576] INFO - rebalance percentage : 16.65 % [2018-01-25 21:44:59,171] - [task:491] INFO - Rebalance - status: running, progress: 16.6471354167 [2018-01-25 21:45:09,419] - [rest_client:1576] INFO - rebalance percentage : 21.60 % [2018-01-25 21:45:09,419] - [task:491] INFO - Rebalance - status: running, progress: 21.6015625 [2018-01-25 21:45:19,507] - [rest_client:1576] INFO - rebalance percentage : 27.01 % [2018-01-25 21:45:19,507] - [task:491] INFO - Rebalance - status: running, progress: 27.0149739583 [2018-01-25 21:45:29,679] - [rest_client:1576] INFO - rebalance percentage : 32.07 % [2018-01-25 21:45:29,679] - [task:491] INFO - Rebalance - status: running, progress: 32.0735677083 [2018-01-25 21:45:39,761] - [rest_client:1576] INFO - rebalance percentage : 37.98 % [2018-01-25 21:45:39,764] - [task:491] INFO - Rebalance - status: running, progress: 37.978515625 [2018-01-25 21:45:49,896] - [rest_client:1576] INFO - rebalance percentage : 42.91 % [2018-01-25 21:45:49,899] - [task:491] INFO - Rebalance - status: running, progress: 42.9134114583 [2018-01-25 21:46:00,390] - [rest_client:1576] INFO - rebalance percentage : 47.81 % [2018-01-25 21:46:00,391] - [task:491] INFO - Rebalance - status: running, progress: 47.8125 [2018-01-25 21:46:10,654] - [rest_client:1576] INFO - rebalance percentage : 54.00 % [2018-01-25 21:46:10,655] - [task:491] INFO - Rebalance - status: running, progress: 54.00390625 [2018-01-25 21:46:20,930] - [rest_client:1576] INFO - rebalance percentage : 55.85 % [2018-01-25 21:46:20,930] - [task:491] INFO - Rebalance - status: running, progress: 55.8463541667 [2018-01-25 21:46:31,020] - [rest_client:1576] INFO - rebalance percentage : 57.14 % [2018-01-25 21:46:31,020] - [task:491] INFO - Rebalance - status: running, progress: 57.1354166667 [2018-01-25 21:46:41,134] - [rest_client:1576] INFO - rebalance percentage : 58.33 % [2018-01-25 21:46:41,135] - [task:491] INFO - Rebalance - status: running, progress: 58.3333333333 [2018-01-25 21:46:51,192] - [rest_client:1576] INFO - rebalance percentage : 58.33 % [2018-01-25 21:46:51,193] - [task:491] INFO - Rebalance - status: running, progress: 58.3333333333 [2018-01-25 21:47:01,252] - [rest_client:1576] INFO - rebalance percentage : 58.33 % [2018-01-25 21:47:01,253] - [task:491] INFO - Rebalance - status: running, progress: 58.3333333333 [2018-01-25 21:47:11,279] - [task:491] INFO - Rebalance - status: none, progress: 100 [2018-01-25 21:47:11,291] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.5:8091/nodes/self error [Errno 111] Connection refused [2018-01-25 21:47:14,294] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.5:8091/nodes/self error [Errno 111] Connection refused [2018-01-25 21:47:17,390] - [task:550] INFO - rebalancing was completed with progress: 100% in 169.169907093 sec [2018-01-25 21:47:17,404] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise'] [2018-01-25 21:47:17,415] - [basetestcase:2242] INFO - list of all nodes in cluster: [] [2018-01-25 21:47:17,428] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise'] [2018-01-25 21:47:17,443] - [basetestcase:2242] INFO - list of all nodes in cluster: [] [2018-01-25 21:47:17,543] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:47:17,634] - [task:1756] INFO - 1000 items will be verified on default bucket [2018-01-25 21:47:18,392] - [task:1765] INFO - 0 items were verified [2018-01-25 21:47:18,477] - [task:1769] INFO - 1000 items were verified in 0.842195987701 sec.the average number of ops - 1187.36938893 per second [2018-01-25 21:47:18,494] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise'] [2018-01-25 21:47:18,500] - [basetestcase:2242] INFO - list of all nodes in cluster: [] [2018-01-25 21:47:19,402] - [task:598] WARNING - Not Ready: curr_items 0 == 1000 expected on , default bucket [2018-01-25 21:47:19,402] - [task:598] WARNING - Not Ready: vb_active_curr_items 0 == 1000 expected on , default bucket [2018-01-25 21:47:19,402] - [task:598] WARNING - Not Ready: vb_replica_curr_items 0 == -1000 expected on , default bucket [2018-01-25 21:47:19,402] - [task:602] INFO - Saw curr_items_tot 0 == 0 expected on ,default bucket [2018-01-25 21:47:24,408] - [task:598] WARNING - Not Ready: curr_items 0 == 1000 expected on , default bucket [2018-01-25 21:47:24,409] - [task:598] WARNING - Not Ready: vb_active_curr_items 0 == 1000 expected on , default bucket [2018-01-25 21:47:24,409] - [task:598] WARNING - Not Ready: vb_replica_curr_items 0 == -1000 expected on , default bucket [2018-01-25 21:47:29,415] - [task:598] WARNING - Not Ready: curr_items 0 == 1000 expected on , default bucket [2018-01-25 21:47:29,415] - [task:598] WARNING - Not Ready: vb_active_curr_items 0 == 1000 expected on , default bucket [2018-01-25 21:47:29,415] - [task:598] WARNING - Not Ready: vb_replica_curr_items 0 == -1000 expected on , default bucket [2018-01-25 21:47:34,420] - [task:598] WARNING - Not Ready: curr_items 0 == 1000 expected on , default bucket [2018-01-25 21:47:34,421] - [task:598] WARNING - Not Ready: vb_active_curr_items 0 == 1000 expected on , default bucket [2018-01-25 21:47:34,421] - [task:598] WARNING - Not Ready: vb_replica_curr_items 0 == -1000 expected on , default bucket [2018-01-25 21:47:39,427] - [task:598] WARNING - Not Ready: curr_items 0 == 1000 expected on , default bucket [2018-01-25 21:47:39,428] - [task:598] WARNING - Not Ready: vb_active_curr_items 0 == 1000 expected on , default bucket [2018-01-25 21:47:39,428] - [task:598] WARNING - Not Ready: vb_replica_curr_items 0 == -1000 expected on , default bucket [2018-01-25 21:47:44,434] - [task:598] WARNING - Not Ready: curr_items 0 == 1000 expected on , default bucket [2018-01-25 21:47:44,434] - [task:598] WARNING - Not Ready: vb_active_curr_items 0 == 1000 expected on , default bucket [2018-01-25 21:47:44,434] - [task:598] WARNING - Not Ready: vb_replica_curr_items 0 == -1000 expected on , default bucket [2018-01-25 21:47:49,439] - [task:598] WARNING - Not Ready: curr_items 0 == 1000 expected on , default bucket [2018-01-25 21:47:49,439] - [task:598] WARNING - Not Ready: vb_active_curr_items 0 == 1000 expected on , default bucket [2018-01-25 21:47:49,439] - [task:598] WARNING - Not Ready: vb_replica_curr_items 0 == -1000 expected on , default bucket [2018-01-25 21:47:54,445] - [task:598] WARNING - Not Ready: curr_items 0 == 1000 expected on , default bucket [2018-01-25 21:47:54,445] - [task:598] WARNING - Not Ready: vb_active_curr_items 0 == 1000 expected on , default bucket [2018-01-25 21:47:54,445] - [task:598] WARNING - Not Ready: vb_replica_curr_items 0 == -1000 expected on , default bucket [2018-01-25 21:47:59,451] - [task:598] WARNING - Not Ready: curr_items 0 == 1000 expected on , default bucket [2018-01-25 21:47:59,451] - [task:598] WARNING - Not Ready: vb_active_curr_items 0 == 1000 expected on , default bucket [2018-01-25 21:47:59,451] - [task:598] WARNING - Not Ready: vb_replica_curr_items 0 == -1000 expected on , default bucket [2018-01-25 21:48:04,456] - [task:598] WARNING - Not Ready: curr_items 0 == 1000 expected on , default bucket [2018-01-25 21:48:04,457] - [task:598] WARNING - Not Ready: vb_active_curr_items 0 == 1000 expected on , default bucket [2018-01-25 21:48:04,457] - [task:598] WARNING - Not Ready: vb_replica_curr_items 0 == -1000 expected on , default bucket [2018-01-25 21:48:09,462] - [task:598] WARNING - Not Ready: curr_items 0 == 1000 expected on , default bucket [2018-01-25 21:48:09,462] - [task:598] WARNING - Not Ready: vb_active_curr_items 0 == 1000 expected on , default bucket [2018-01-25 21:48:09,462] - [task:598] WARNING - Not Ready: vb_replica_curr_items 0 == -1000 expected on , default bucket [2018-01-25 21:48:14,469] - [task:598] WARNING - Not Ready: curr_items 0 == 1000 expected on , default bucket [2018-01-25 21:48:14,469] - [task:598] WARNING - Not Ready: vb_active_curr_items 0 == 1000 expected on , default bucket [2018-01-25 21:48:14,469] - [task:598] WARNING - Not Ready: vb_replica_curr_items 0 == -1000 expected on , default bucket [2018-01-25 21:48:19,475] - [task:598] WARNING - Not Ready: curr_items 0 == 1000 expected on , default bucket [2018-01-25 21:48:19,475] - [task:598] WARNING - Not Ready: vb_active_curr_items 0 == 1000 expected on , default bucket [2018-01-25 21:48:19,475] - [task:598] WARNING - Not Ready: vb_replica_curr_items 0 == -1000 expected on , default bucket [2018-01-25 21:48:24,482] - [task:598] WARNING - Not Ready: curr_items 0 == 1000 expected on , default bucket [2018-01-25 21:48:24,484] - [task:598] WARNING - Not Ready: vb_active_curr_items 0 == 1000 expected on , default bucket [2018-01-25 21:48:24,484] - [task:598] WARNING - Not Ready: vb_replica_curr_items 0 == -1000 expected on , default bucket [2018-01-25 21:48:29,493] - [task:598] WARNING - Not Ready: curr_items 0 == 1000 expected on , default bucket [2018-01-25 21:48:29,493] - [task:598] WARNING - Not Ready: vb_active_curr_items 0 == 1000 expected on , default bucket [2018-01-25 21:48:29,493] - [task:598] WARNING - Not Ready: vb_replica_curr_items 0 == -1000 expected on , default bucket [2018-01-25 21:48:34,501] - [task:598] WARNING - Not Ready: curr_items 0 == 1000 expected on , default bucket [2018-01-25 21:48:34,501] - [task:598] WARNING - Not Ready: vb_active_curr_items 0 == 1000 expected on , default bucket [2018-01-25 21:48:34,501] - [task:598] WARNING - Not Ready: vb_replica_curr_items 0 == -1000 expected on , default bucket [2018-01-25 21:48:39,508] - [task:598] WARNING - Not Ready: curr_items 0 == 1000 expected on , default bucket [2018-01-25 21:48:39,508] - [task:598] WARNING - Not Ready: vb_active_curr_items 0 == 1000 expected on , default bucket [2018-01-25 21:48:39,508] - [task:598] WARNING - Not Ready: vb_replica_curr_items 0 == -1000 expected on , default bucket [2018-01-25 21:48:44,515] - [task:598] WARNING - Not Ready: curr_items 0 == 1000 expected on , default bucket [2018-01-25 21:48:44,516] - [task:598] WARNING - Not Ready: vb_active_curr_items 0 == 1000 expected on , default bucket [2018-01-25 21:48:44,516] - [task:598] WARNING - Not Ready: vb_replica_curr_items 0 == -1000 expected on , default bucket [2018-01-25 21:48:49,520] - [task:598] WARNING - Not Ready: curr_items 0 == 1000 expected on , default bucket [2018-01-25 21:48:49,521] - [task:598] WARNING - Not Ready: vb_active_curr_items 0 == 1000 expected on , default bucket [2018-01-25 21:48:49,521] - [task:598] WARNING - Not Ready: vb_replica_curr_items 0 == -1000 expected on , default bucket [2018-01-25 21:48:54,527] - [task:598] WARNING - Not Ready: curr_items 0 == 1000 expected on , default bucket [2018-01-25 21:48:54,527] - [task:598] WARNING - Not Ready: vb_active_curr_items 0 == 1000 expected on , default bucket [2018-01-25 21:48:54,528] - [task:598] WARNING - Not Ready: vb_replica_curr_items 0 == -1000 expected on , default bucket [2018-01-25 21:48:59,535] - [task:598] WARNING - Not Ready: curr_items 0 == 1000 expected on , default bucket [2018-01-25 21:48:59,535] - [task:598] WARNING - Not Ready: vb_active_curr_items 0 == 1000 expected on , default bucket [2018-01-25 21:48:59,536] - [task:598] WARNING - Not Ready: vb_replica_curr_items 0 == -1000 expected on , default bucket [2018-01-25 21:49:04,541] - [task:598] WARNING - Not Ready: curr_items 0 == 1000 expected on , default bucket [2018-01-25 21:49:04,541] - [task:598] WARNING - Not Ready: vb_active_curr_items 0 == 1000 expected on , default bucket [2018-01-25 21:49:04,542] - [task:598] WARNING - Not Ready: vb_replica_curr_items 0 == -1000 expected on , default bucket [2018-01-25 21:49:09,547] - [task:598] WARNING - Not Ready: curr_items 0 == 1000 expected on , default bucket [2018-01-25 21:49:09,547] - [task:598] WARNING - Not Ready: vb_active_curr_items 0 == 1000 expected on , default bucket [2018-01-25 21:49:09,547] - [task:598] WARNING - Not Ready: vb_replica_curr_items 0 == -1000 expected on , default bucket [2018-01-25 21:49:14,552] - [task:598] WARNING - Not Ready: curr_items 0 == 1000 expected on , default bucket [2018-01-25 21:49:14,553] - [task:598] WARNING - Not Ready: vb_active_curr_items 0 == 1000 expected on , default bucket [2018-01-25 21:49:14,553] - [task:598] WARNING - Not Ready: vb_replica_curr_items 0 == -1000 expected on , default bucket [('/usr/lib/python2.7/threading.py', 783, '__bootstrap', 'self.__bootstrap_inner()'), ('/usr/lib/python2.7/threading.py', 810, '__bootstrap_inner', 'self.run()'), ('./testrunner.py', 297, 'run', '**self._Thread__kwargs)'), ('/usr/lib/python2.7/unittest/runner.py', 151, 'run', 'test(result)'), ('/usr/lib/python2.7/unittest/case.py', 395, '__call__', 'return self.run(*args, **kwds)'), ('/usr/lib/python2.7/unittest/case.py', 331, 'run', 'testMethod()'), ('pytests/rebalance/rebalanceout.py', 186, 'rebalance_out_with_ops', 'self.verify_cluster_stats(self.servers[:self.num_servers - self.nodes_out])'), ('pytests/basetestcase.py', 1143, 'verify_cluster_stats', 'self._verify_stats_all_buckets(servers, timeout=(timeout or 120))'), ('pytests/basetestcase.py', 815, '_verify_stats_all_buckets', 'task.result(timeout)'), ('lib/tasks/future.py', 162, 'result', 'self.set_exception(TimeoutError())'), ('lib/tasks/future.py', 264, 'set_exception', 'print traceback.extract_stack()')] Thu Jan 25 21:49:18 2018 [2018-01-25 21:49:18,502] - [basetestcase:817] INFO - [2018-01-25 21:49:18,502] - [basetestcase:820] ERROR - unable to get expected stats for any node! Print taps for all nodes: [2018-01-25 21:49:18,529] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:49:18,578] - [rebalance_helper:317] INFO - getting tap stats... for cb-example-0003.cb-example.default.svc rebalance_out_with_ops (rebalance.rebalanceout.RebalanceOutTests) ... ERROR [2018-01-25 21:49:18,578] - [rebalance_helper:323] ERROR - error Memcached error #1 'Not found': for vbucket :0 to mc cb-example-0003.cb-example.default.svc:11210 while getting stats... [2018-01-25 21:49:18,580] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 21:49:18,663] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 21:49:18,991] - [remote_util:2995] INFO - running command.raw on 172.17.0.8: kill -9 $(ps aux | pgrep 'memcached') [2018-01-25 21:49:19,067] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:49:19,069] - [remote_util:232] INFO - connecting to 172.17.0.6 with username:root password:couchbase ssh_key: [2018-01-25 21:49:19,173] - [remote_util:266] INFO - Connected to 172.17.0.6 [2018-01-25 21:49:19,660] - [remote_util:2995] INFO - running command.raw on 172.17.0.6: kill -9 $(ps aux | pgrep 'memcached') [2018-01-25 21:49:19,747] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:49:19,877] - [remote_util:232] INFO - connecting to 172.17.0.7 with username:root password:couchbase ssh_key: [2018-01-25 21:49:20,014] - [remote_util:266] INFO - Connected to 172.17.0.7 [2018-01-25 21:49:20,483] - [remote_util:2995] INFO - running command.raw on 172.17.0.7: kill -9 $(ps aux | pgrep 'memcached') [2018-01-25 21:49:20,557] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:49:20,673] - [remote_util:232] INFO - connecting to 172.17.0.5 with username:root password:couchbase ssh_key: [2018-01-25 21:49:20,756] - [remote_util:266] INFO - Connected to 172.17.0.5 [2018-01-25 21:49:21,131] - [remote_util:2995] INFO - running command.raw on 172.17.0.5: kill -9 $(ps aux | pgrep 'memcached') [2018-01-25 21:49:21,203] - [remote_util:3032] INFO - command executed successfully ====================================================================== ERROR: rebalance_out_with_ops (rebalance.rebalanceout.RebalanceOutTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "pytests/rebalance/rebalanceout.py", line 186, in rebalance_out_with_ops self.verify_cluster_stats(self.servers[:self.num_servers - self.nodes_out]) File "pytests/basetestcase.py", line 1143, in verify_cluster_stats self._verify_stats_all_buckets(servers, timeout=(timeout or 120)) File "pytests/basetestcase.py", line 824, in _verify_stats_all_buckets raise Exception("unable to get expected stats during {0} sec".format(timeout)) Exception: unable to get expected stats during 120 sec ---------------------------------------------------------------------- Ran 1 test in 414.429s FAILED (errors=1) Cluster instance shutdown with force summary so far suite recreatebuckettests.RecreateMembaseBuckets , pass 0 , fail 1 failures so far... recreatebuckettests.RecreateMembaseBuckets.test_default_moxi summary so far suite deletebuckettests.DeleteMembaseBuckets , pass 1 , fail 0 summary so far suite createbuckettests.CreateMembaseBucketsTests , pass 1 , fail 4 failures so far... createbuckettests.CreateMembaseBucketsTests.test_default_moxi createbuckettests.CreateMembaseBucketsTests.test_default_on_non_default_port createbuckettests.CreateMembaseBucketsTests.test_non_default_case_sensitive_different_port createbuckettests.CreateMembaseBucketsTests.test_two_replica summary so far suite setgettests.MembaseBucket , pass 1 , fail 0 summary so far suite expirytests.ExpiryTests , pass 1 , fail 0 summary so far suite memcapable.GetlTests , pass 2 , fail 0 summary so far suite memorysanitytests.MemorySanity , pass 1 , fail 0 summary so far suite drainratetests.DrainRateTests , pass 1 , fail 0 summary so far suite view.viewquerytests.ViewQueryTests , pass 1 , fail 0 summary so far suite view.createdeleteview.CreateDeleteViewTests , pass 1 , fail 0 summary so far suite rebalance.rebalancein.RebalanceInTests , pass 0 , fail 1 failures so far... rebalance.rebalancein.RebalanceInTests.rebalance_in_with_ops summary so far suite rebalance.rebalanceout.RebalanceOutTests , pass 0 , fail 1 failures so far... rebalance.rebalanceout.RebalanceOutTests.rebalance_out_with_ops testrunner logs, diags and results are available under /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_17 Logs will be stored at /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_18 ./testrunner -i ./4node.ini -t failover.failovertests.FailoverTests.test_failover_normal,replica=1,load_ratio=1,num_failed_nodes=1,withMutationOps=True Test Input params: {'num_failed_nodes': '1', 'conf_file': './4node.conf', 'num_nodes': 4, 'cluster_name': '4node', 'replica': '1', 'ini': './4node.ini', 'case_number': 18, 'load_ratio': '1', 'logs_folder': '/testrunner/logs/testrunner-18-Jan-25_21-16-28/test_18', 'withMutationOps': 'True', 'spec': '4node'} [2018-01-25 21:49:21,340] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 21:49:21,424] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 21:49:21,807] - [rest_client:47] WARNING - server 172.17.0.8:8091 status is warmup [2018-01-25 21:49:23,813] - [rest_client:47] WARNING - server 172.17.0.8:8091 status is warmup [2018-01-25 21:49:25,827] - [rest_client:1964] INFO - Node version in cluster 5.0.1-5003-enterprise [2018-01-25 21:49:25,837] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise'] [2018-01-25 21:49:25,838] - [basetestcase:201] INFO - ============== basetestcase setup was started for test #18 test_failover_normal============== [2018-01-25 21:49:25,860] - [bucket_helper:142] INFO - deleting existing buckets [u'default'] on 172.17.0.8 [2018-01-25 21:49:25,860] - [bucket_helper:144] INFO - remove bucket default ... [2018-01-25 21:49:26,305] - [bucket_helper:158] INFO - deleted bucket : default from 172.17.0.8 [2018-01-25 21:49:26,305] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:49:26,308] - [rest_client:133] INFO - node 172.17.0.8 existing buckets : [] [2018-01-25 21:49:26,308] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:49:28,316] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.6 [2018-01-25 21:49:28,316] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:49:30,342] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.7 [2018-01-25 21:49:30,343] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:49:32,350] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.5 [2018-01-25 21:49:32,350] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:49:34,376] - [basetestcase:482] INFO - sleep for 10 secs. ... [2018-01-25 21:49:44,384] - [cluster_helper:78] INFO - waiting for ns_server @ 172.17.0.8:8091 [2018-01-25 21:49:44,388] - [cluster_helper:80] INFO - ns_server @ 172.17.0.8:8091 is running [2018-01-25 21:49:44,392] - [cluster_helper:78] INFO - waiting for ns_server @ 172.17.0.6:8091 [2018-01-25 21:49:44,395] - [cluster_helper:80] INFO - ns_server @ 172.17.0.6:8091 is running [2018-01-25 21:49:44,398] - [cluster_helper:78] INFO - waiting for ns_server @ 172.17.0.7:8091 [2018-01-25 21:49:44,401] - [cluster_helper:80] INFO - ns_server @ 172.17.0.7:8091 is running [2018-01-25 21:49:44,406] - [cluster_helper:78] INFO - waiting for ns_server @ 172.17.0.5:8091 [2018-01-25 21:49:44,408] - [cluster_helper:80] INFO - ns_server @ 172.17.0.5:8091 is running [2018-01-25 21:49:44,409] - [basetestcase:245] INFO - initializing cluster [2018-01-25 21:49:45,374] - [task:125] INFO - server: ip:172.17.0.8 port:8091 ssh_username:root, nodes/self: {'ip': u'cb-example-0003.cb-example.default.svc', 'availableStorage': [], 'rest_username': '', 'id': u'ns_1@cb-example-0003.cb-example.default.svc', 'uptime': u'2232', 'mcdMemoryReserved': 6234, 'storageTotalRam': 7793, 'hostname': u'cb-example-0003.cb-example.default.svc:8091', 'storage': [], 'moxi': 11211, 'port': u'8091', 'version': u'5.0.1-5003-enterprise', 'memcached': 11210, 'status': u'healthy', 'clusterCompatibility': 327680, 'curr_items': 0, 'services': [u'fts', u'index', u'kv', u'n1ql'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 5661663232, 'memoryTotal': 8171810816, 'memoryQuota': 3900, 'mcdMemoryAllocated': 6234, 'os': u'x86_64-unknown-linux-gnu', 'ports': []} [2018-01-25 21:49:45,375] - [task:157] INFO - quota for index service will be 256 MB [2018-01-25 21:49:45,375] - [task:159] INFO - set index quota to node 172.17.0.8 [2018-01-25 21:49:45,375] - [rest_client:919] INFO - pools/default params : indexMemoryQuota=256 [2018-01-25 21:49:45,386] - [rest_client:907] INFO - pools/default params : memoryQuota=3900 [2018-01-25 21:49:45,400] - [rest_client:942] INFO - settings/indexes params : storageMode=plasma [2018-01-25 21:49:45,404] - [rest_client:807] ERROR - POST http://172.17.0.8:8091/settings/indexes body: storageMode=plasma headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 400 reason: unknown {"errors":{"storageMode":"Changing the optimization mode of global indexes is not supported when index service nodes are present in the cluster. Please remove all index service nodes to change this option."}} auth: Administrator:password [2018-01-25 21:49:45,404] - [rest_client:885] INFO - /node/controller/setupServices params on 172.17.0.8: 8091:services=n1ql%2Ckv%2Cindex&password=password&hostname=172.17.0.8&user=Administrator [2018-01-25 21:49:45,407] - [rest_client:807] ERROR - POST http://172.17.0.8:8091//node/controller/setupServices body: services=n1ql%2Ckv%2Cindex&password=password&hostname=172.17.0.8&user=Administrator headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 400 reason: unknown ["cannot change node services after cluster is provisioned"] auth: Administrator:password [2018-01-25 21:49:45,407] - [rest_client:890] INFO - This node is already provisioned with services, we do not consider this as failure for test case [2018-01-25 21:49:45,407] - [rest_client:825] INFO - settings/web params on 172.17.0.8:8091:username=Administrator&password=password&port=8091 [2018-01-25 21:49:45,486] - [task:125] INFO - server: ip:172.17.0.6 port:8091 ssh_username:root, nodes/self: {'ip': u'172.17.0.6', 'availableStorage': [], 'rest_username': '', 'id': u'ns_1@172.17.0.6', 'uptime': u'2258', 'mcdMemoryReserved': 6234, 'storageTotalRam': 7793, 'hostname': u'172.17.0.6:8091', 'storage': [], 'moxi': 11211, 'port': u'8091', 'version': u'5.0.1-5003-enterprise', 'memcached': 11210, 'status': u'healthy', 'clusterCompatibility': 327680, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 5655056384, 'memoryTotal': 8171810816, 'memoryQuota': 4675, 'mcdMemoryAllocated': 6234, 'os': u'x86_64-unknown-linux-gnu', 'ports': []} [2018-01-25 21:49:45,487] - [rest_client:907] INFO - pools/default params : memoryQuota=4156 [2018-01-25 21:49:45,492] - [rest_client:942] INFO - settings/indexes params : storageMode=plasma [2018-01-25 21:49:45,497] - [rest_client:825] INFO - settings/web params on 172.17.0.6:8091:username=Administrator&password=password&port=8091 [2018-01-25 21:49:45,574] - [task:125] INFO - server: ip:172.17.0.7 port:8091 ssh_username:root, nodes/self: {'ip': u'172.17.0.7', 'availableStorage': [], 'rest_username': '', 'id': u'ns_1@172.17.0.7', 'uptime': u'2243', 'mcdMemoryReserved': 6234, 'storageTotalRam': 7793, 'hostname': u'172.17.0.7:8091', 'storage': [], 'moxi': 11211, 'port': u'8091', 'version': u'5.0.1-5003-enterprise', 'memcached': 11210, 'status': u'healthy', 'clusterCompatibility': 327680, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 5647814656, 'memoryTotal': 8171810816, 'memoryQuota': 4675, 'mcdMemoryAllocated': 6234, 'os': u'x86_64-unknown-linux-gnu', 'ports': []} [2018-01-25 21:49:45,575] - [rest_client:907] INFO - pools/default params : memoryQuota=4156 [2018-01-25 21:49:45,580] - [rest_client:942] INFO - settings/indexes params : storageMode=plasma [2018-01-25 21:49:45,586] - [rest_client:825] INFO - settings/web params on 172.17.0.7:8091:username=Administrator&password=password&port=8091 [2018-01-25 21:49:45,673] - [task:125] INFO - server: ip:172.17.0.5 port:8091 ssh_username:root, nodes/self: {'ip': u'172.17.0.5', 'availableStorage': [], 'rest_username': '', 'id': u'ns_1@172.17.0.5', 'uptime': u'2280', 'mcdMemoryReserved': 6234, 'storageTotalRam': 7793, 'hostname': u'172.17.0.5:8091', 'storage': [], 'moxi': 11211, 'port': u'8091', 'version': u'5.0.1-5003-enterprise', 'memcached': 11210, 'status': u'healthy', 'clusterCompatibility': 327680, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 5647314944, 'memoryTotal': 8171810816, 'memoryQuota': 4675, 'mcdMemoryAllocated': 6234, 'os': u'x86_64-unknown-linux-gnu', 'ports': []} [2018-01-25 21:49:45,674] - [rest_client:907] INFO - pools/default params : memoryQuota=4156 [2018-01-25 21:49:45,680] - [rest_client:942] INFO - settings/indexes params : storageMode=plasma [2018-01-25 21:49:45,685] - [rest_client:825] INFO - settings/web params on 172.17.0.5:8091:username=Administrator&password=password&port=8091 [2018-01-25 21:49:45,911] - [basetestcase:2060] INFO - **** add built-in 'cbadminbucket' user to node 172.17.0.8 **** [2018-01-25 21:49:46,102] - [basetestcase:482] INFO - sleep for 10 secs. ... [2018-01-25 21:49:56,112] - [basetestcase:2065] INFO - **** add 'admin' role to 'cbadminbucket' user **** [2018-01-25 21:49:56,126] - [basetestcase:482] INFO - sleep for 10 secs. ... [2018-01-25 21:50:06,136] - [basetestcase:269] INFO - done initializing cluster [2018-01-25 21:50:06,902] - [task:425] INFO - adding node 172.17.0.6:8091 to cluster [2018-01-25 21:50:06,902] - [rest_client:1205] INFO - adding remote node @172.17.0.6:8091 to this cluster @172.17.0.8:8091 [2018-01-25 21:50:12,594] - [task:425] INFO - adding node 172.17.0.7:8091 to cluster [2018-01-25 21:50:12,607] - [rest_client:1205] INFO - adding remote node @172.17.0.7:8091 to this cluster @172.17.0.8:8091 [2018-01-25 21:50:16,130] - [task:425] INFO - adding node 172.17.0.5:8091 to cluster [2018-01-25 21:50:16,130] - [rest_client:1205] INFO - adding remote node @172.17.0.5:8091 to this cluster @172.17.0.8:8091 [2018-01-25 21:50:20,521] - [rest_client:1423] INFO - rebalance params : {'password': 'password', 'ejectedNodes': '', 'user': 'Administrator', 'knownNodes': u'ns_1@cb-example-0003.cb-example.default.svc,ns_1@172.17.0.7,ns_1@172.17.0.5,ns_1@172.17.0.6'} [2018-01-25 21:50:20,532] - [rest_client:1428] INFO - rebalance operation started [2018-01-25 21:50:20,570] - [rest_client:1576] INFO - rebalance percentage : 0.00 % [2018-01-25 21:50:20,571] - [task:491] INFO - Rebalance - status: running, progress: 0 [2018-01-25 21:50:30,647] - [task:491] INFO - Rebalance - status: none, progress: 100 [2018-01-25 21:50:30,652] - [task:550] INFO - rebalancing was completed with progress: 100% in 10.11992383 sec [2018-01-25 21:50:31,684] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:50:31,685] - [rest_client:2237] INFO - http://172.17.0.8:8091/pools/default/buckets with param: bucketType=membase&threadsNumber=3&authType=sasl&replicaIndex=1&name=default&saslPassword=None&evictionPolicy=valueOnly&flushEnabled=1&replicaNumber=1&ramQuotaMB=3900 [2018-01-25 21:50:31,695] - [rest_client:2262] INFO - 0.01 seconds to create bucket default [2018-01-25 21:50:31,695] - [bucket_helper:343] INFO - waiting for memcached bucket : default in 172.17.0.8 to accept set ops [2018-01-25 21:50:33,934] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 default [2018-01-25 21:50:35,138] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 default [2018-01-25 21:50:36,816] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 default [2018-01-25 21:50:38,111] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:50:38,716] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 default [2018-01-25 21:50:38,877] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 default [2018-01-25 21:50:39,418] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 default [2018-01-25 21:50:40,173] - [data_helper:295] INFO - creating direct client cb-example-0003.cb-example.default.svc:11210 default [2018-01-25 21:50:40,894] - [task:312] INFO - bucket 'default' was created with per node RAM quota: 3900 [2018-01-25 21:50:40,932] - [basetestcase:344] INFO - ============== basetestcase setup was finished for test #18 test_failover_normal ============== [2018-01-25 21:50:40,938] - [basetestcase:482] INFO - sleep for 10 secs. ... [2018-01-25 21:50:50,948] - [failoverbasetests:67] INFO - ============== FailoverBaseTest setup was finished for test #18 test_failover_normal ============== [2018-01-25 21:50:50,949] - [failovertests:46] INFO - Picking node 172.17.0.8 as reference node for test case [2018-01-25 21:50:50,949] - [failovertests:264] INFO - num_replicas : 1 [2018-01-25 21:50:50,949] - [failovertests:265] INFO - recoveryType : ['delta'] [2018-01-25 21:50:50,949] - [failovertests:266] INFO - failover_reason : normal [2018-01-25 21:50:50,949] - [failovertests:267] INFO - num_failed_nodes : 1 [2018-01-25 21:50:50,949] - [failovertests:268] INFO - picking server : ip:172.17.0.8 port:8091 ssh_username:root as the master [2018-01-25 21:50:50,980] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:50:50,980] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:50:51,037] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:50:51,052] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 default [2018-01-25 21:50:51,097] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:50:51,103] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:50:51,116] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 default [2018-01-25 21:50:51,172] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:50:51,178] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:50:51,186] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 default [2018-01-25 21:50:51,249] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:50:52,013] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:50:52,029] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:50:52,035] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:50:52,044] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 default [2018-01-25 21:50:52,045] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:50:52,051] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 default [2018-01-25 21:50:52,064] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 default [2018-01-25 21:50:52,064] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 default [2018-01-25 21:50:52,120] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:50:52,157] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:50:52,166] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:50:52,169] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 default [2018-01-25 21:50:52,173] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:50:52,174] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:50:52,183] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:50:52,196] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:50:52,199] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 default [2018-01-25 21:50:52,200] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:50:52,225] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 default [2018-01-25 21:50:52,227] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 default [2018-01-25 21:50:52,284] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:50:52,295] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:50:52,300] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:50:52,311] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:50:52,321] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 default [2018-01-25 21:50:52,322] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:50:52,336] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 default [2018-01-25 21:50:52,338] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:50:52,341] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:50:52,357] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 default [2018-01-25 21:50:52,360] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:50:52,384] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 default [2018-01-25 21:50:52,405] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:50:52,406] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:50:52,506] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:50:52,777] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:50:53,549] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:50:53,553] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:50:53,595] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:50:53,605] - [basetestcase:2242] INFO - list of all nodes in cluster: [ip:172.17.0.7 port:8091 ssh_username:root, ip:172.17.0.5 port:8091 ssh_username:root, ip:172.17.0.6 port:8091 ssh_username:root] [2018-01-25 21:50:53,945] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 default [2018-01-25 21:50:53,984] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:50:53,988] - [task:602] INFO - Saw ep_queue_size 0 == 0 expected on '172.17.0.6:8091',default bucket [2018-01-25 21:50:53,997] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 default [2018-01-25 21:50:54,037] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:50:54,084] - [task:602] INFO - Saw ep_dcp_items_remaining 0 == 0 expected on '172.17.0.6:8091',default bucket [2018-01-25 21:50:54,094] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 default [2018-01-25 21:50:54,132] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:50:54,139] - [task:602] INFO - Saw ep_queue_size 0 == 0 expected on '172.17.0.7:8091',default bucket [2018-01-25 21:50:54,152] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 default [2018-01-25 21:50:54,189] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:50:54,240] - [task:602] INFO - Saw ep_dcp_items_remaining 0 == 0 expected on '172.17.0.7:8091',default bucket [2018-01-25 21:50:54,249] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 default [2018-01-25 21:50:54,283] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:50:54,288] - [task:602] INFO - Saw ep_queue_size 0 == 0 expected on '172.17.0.5:8091',default bucket [2018-01-25 21:50:54,295] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 default [2018-01-25 21:50:54,332] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:50:54,378] - [task:602] INFO - Saw ep_dcp_items_remaining 0 == 0 expected on '172.17.0.5:8091',default bucket [2018-01-25 21:50:54,424] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:50:54,425] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:50:54,432] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:50:54,433] - [basetestcase:2242] INFO - list of all nodes in cluster: [ip:172.17.0.7 port:8091 ssh_username:root, ip:172.17.0.5 port:8091 ssh_username:root, ip:172.17.0.6 port:8091 ssh_username:root] [2018-01-25 21:50:55,390] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 default [2018-01-25 21:50:55,464] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:50:55,510] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 default [2018-01-25 21:50:55,637] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:50:55,653] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 default [2018-01-25 21:50:55,697] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:50:55,704] - [task:602] INFO - Saw curr_items 1000 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091',default bucket [2018-01-25 21:50:55,712] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 default [2018-01-25 21:50:55,753] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:50:55,771] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 default [2018-01-25 21:50:55,820] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:50:55,859] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 default [2018-01-25 21:50:55,910] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:50:55,917] - [task:602] INFO - Saw vb_active_curr_items 1000 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091',default bucket [2018-01-25 21:50:55,926] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 default [2018-01-25 21:50:55,982] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:50:56,004] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 default [2018-01-25 21:50:56,054] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:50:56,072] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 default [2018-01-25 21:50:56,112] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:50:56,120] - [task:598] WARNING - Not Ready: vb_replica_curr_items 493 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:50:56,131] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 default [2018-01-25 21:50:56,179] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:50:56,195] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 default [2018-01-25 21:50:56,255] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:50:56,271] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 default [2018-01-25 21:50:56,322] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:50:56,327] - [task:598] WARNING - Not Ready: curr_items_tot 1493 == 2000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:51:01,347] - [task:598] WARNING - Not Ready: vb_replica_curr_items 493 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:51:01,359] - [task:598] WARNING - Not Ready: curr_items_tot 1493 == 2000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:51:06,375] - [task:598] WARNING - Not Ready: vb_replica_curr_items 493 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:51:06,389] - [task:598] WARNING - Not Ready: curr_items_tot 1493 == 2000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:51:12,087] - [task:598] WARNING - Not Ready: vb_replica_curr_items 493 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:51:12,104] - [task:598] WARNING - Not Ready: curr_items_tot 1493 == 2000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:51:17,143] - [task:598] WARNING - Not Ready: vb_replica_curr_items 493 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:51:17,169] - [task:598] WARNING - Not Ready: curr_items_tot 1493 == 2000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:51:22,287] - [task:598] WARNING - Not Ready: vb_replica_curr_items 493 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:51:22,328] - [task:598] WARNING - Not Ready: curr_items_tot 1493 == 2000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:51:27,342] - [task:598] WARNING - Not Ready: vb_replica_curr_items 493 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:51:27,353] - [task:598] WARNING - Not Ready: curr_items_tot 1493 == 2000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:51:32,392] - [task:598] WARNING - Not Ready: vb_replica_curr_items 493 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:51:32,413] - [task:598] WARNING - Not Ready: curr_items_tot 1493 == 2000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:51:37,427] - [task:598] WARNING - Not Ready: vb_replica_curr_items 493 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:51:37,437] - [task:598] WARNING - Not Ready: curr_items_tot 1493 == 2000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:51:43,023] - [task:598] WARNING - Not Ready: vb_replica_curr_items 493 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:51:43,064] - [task:598] WARNING - Not Ready: curr_items_tot 1493 == 2000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:51:48,101] - [task:598] WARNING - Not Ready: vb_replica_curr_items 493 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:51:48,127] - [task:598] WARNING - Not Ready: curr_items_tot 1493 == 2000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:51:53,345] - [task:598] WARNING - Not Ready: vb_replica_curr_items 493 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:51:53,407] - [task:598] WARNING - Not Ready: curr_items_tot 1493 == 2000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:51:58,436] - [task:598] WARNING - Not Ready: vb_replica_curr_items 493 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:51:58,467] - [task:598] WARNING - Not Ready: curr_items_tot 1493 == 2000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:52:04,218] - [task:598] WARNING - Not Ready: vb_replica_curr_items 493 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:52:04,243] - [task:598] WARNING - Not Ready: curr_items_tot 1493 == 2000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:52:09,286] - [task:598] WARNING - Not Ready: vb_replica_curr_items 493 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:52:09,305] - [task:598] WARNING - Not Ready: curr_items_tot 1493 == 2000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:52:15,039] - [task:598] WARNING - Not Ready: vb_replica_curr_items 493 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:52:15,197] - [task:598] WARNING - Not Ready: curr_items_tot 1493 == 2000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:52:20,213] - [task:598] WARNING - Not Ready: vb_replica_curr_items 493 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:52:20,223] - [task:598] WARNING - Not Ready: curr_items_tot 1493 == 2000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:52:25,251] - [task:598] WARNING - Not Ready: vb_replica_curr_items 493 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:52:25,271] - [task:598] WARNING - Not Ready: curr_items_tot 1493 == 2000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:52:30,287] - [task:598] WARNING - Not Ready: vb_replica_curr_items 493 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:52:30,297] - [task:598] WARNING - Not Ready: curr_items_tot 1493 == 2000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:52:35,319] - [task:598] WARNING - Not Ready: vb_replica_curr_items 493 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:52:35,330] - [task:598] WARNING - Not Ready: curr_items_tot 1493 == 2000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:52:40,346] - [task:598] WARNING - Not Ready: vb_replica_curr_items 493 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:52:40,356] - [task:598] WARNING - Not Ready: curr_items_tot 1493 == 2000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:52:45,461] - [task:598] WARNING - Not Ready: vb_replica_curr_items 493 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:52:45,471] - [task:598] WARNING - Not Ready: curr_items_tot 1493 == 2000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:52:50,493] - [task:598] WARNING - Not Ready: vb_replica_curr_items 493 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:52:50,503] - [task:598] WARNING - Not Ready: curr_items_tot 1493 == 2000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:52:55,518] - [task:598] WARNING - Not Ready: vb_replica_curr_items 493 == 1000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [2018-01-25 21:52:55,529] - [task:598] WARNING - Not Ready: curr_items_tot 1493 == 2000 expected on '172.17.0.6:8091''172.17.0.7:8091''172.17.0.5:8091', default bucket [('/usr/lib/python2.7/threading.py', 783, '__bootstrap', 'self.__bootstrap_inner()'), ('/usr/lib/python2.7/threading.py', 810, '__bootstrap_inner', 'self.run()'), ('./testrunner.py', 297, 'run', '**self._Thread__kwargs)'), ('/usr/lib/python2.7/unittest/runner.py', 151, 'run', 'test(result)'), ('/usr/lib/python2.7/unittest/case.py', 395, '__call__', 'return self.run(*args, **kwds)'), ('/usr/lib/python2.7/unittest/case.py', 331, 'run', 'testMethod()'), ('pytests/failover/failovertests.py', 22, 'test_failover_normal', "self.common_test_body('normal')"), ('pytests/failover/failovertests.py', 75, 'common_test_body', 'self.load_initial_data()'), ('pytests/failover/failovertests.py', 426, 'load_initial_data', 'self._verify_stats_all_buckets(self.servers, timeout=120)'), ('pytests/basetestcase.py', 815, '_verify_stats_all_buckets', 'task.result(timeout)'), ('lib/tasks/future.py', 162, 'result', 'self.set_exception(TimeoutError())'), ('lib/tasks/future.py', 264, 'set_exception', 'print traceback.extract_stack()')] Thu Jan 25 21:52:55 2018 [2018-01-25 21:52:55,947] - [basetestcase:817] INFO - [2018-01-25 21:52:55,947] - [basetestcase:820] ERROR - unable to get expected stats for any node! Print taps for all nodes: [2018-01-25 21:52:55,967] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:52:55,983] - [data_helper:295] INFO - creating direct client 172.17.0.5:11210 default [2018-01-25 21:52:56,061] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:52:56,062] - [rebalance_helper:317] INFO - getting tap stats... for 172.17.0.5 [2018-01-25 21:52:56,063] - [rebalance_helper:323] ERROR - error Memcached error #1 'Not found': for vbucket :0 to mc 172.17.0.5:11210 while getting stats... [2018-01-25 21:52:56,085] - [data_helper:295] INFO - creating direct client 172.17.0.6:11210 default [2018-01-25 21:52:56,182] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:52:56,184] - [rebalance_helper:317] INFO - getting tap stats... for 172.17.0.6 [2018-01-25 21:52:56,187] - [rebalance_helper:323] ERROR - error Memcached error #1 'Not found': for vbucket :0 to mc 172.17.0.6:11210 while getting stats... [2018-01-25 21:52:56,215] - [data_helper:295] INFO - creating direct client 172.17.0.7:11210 default [2018-01-25 21:52:56,296] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:52:56,297] - [rebalance_helper:317] INFO - getting tap stats... for 172.17.0.7 [2018-01-25 21:52:56,298] - [rebalance_helper:323] ERROR - error Memcached error #1 'Not found': for vbucket :0 to mc 172.17.0.7:11210 while getting stats... [2018-01-25 21:52:56,298] - [failoverbasetests:81] INFO - ============== tearDown was started for test #18 test_failover_normal ============== test_failover_normal (failover.failovertests.FailoverTests) ... ERROR [2018-01-25 21:52:56,300] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 21:52:57,106] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 21:52:57,675] - [remote_util:2995] INFO - running command.raw on 172.17.0.8: /etc/init.d/couchbase-server start [2018-01-25 21:52:57,684] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:52:57,684] - [remote_util:2830] INFO - * couchbase-server is already started [2018-01-25 21:52:57,684] - [remote_util:2995] INFO - running command.raw on 172.17.0.8: /sbin/iptables -F [2018-01-25 21:52:57,753] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:52:57,753] - [remote_util:2793] ERROR - bash: /sbin/iptables: No such file or directory [2018-01-25 21:52:57,753] - [remote_util:2995] INFO - running command.raw on 172.17.0.8: /sbin/iptables -t nat -F [2018-01-25 21:52:57,824] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:52:57,824] - [remote_util:2793] ERROR - bash: /sbin/iptables: No such file or directory [2018-01-25 21:52:57,824] - [remote_util:291] INFO - Connect to node: 172.17.0.8 as user: root [2018-01-25 21:52:57,961] - [remote_util:314] INFO - Connected to 172.17.0.8 as root [2018-01-25 21:52:57,962] - [remote_util:3450] INFO - *** unpause memcached process *** [2018-01-25 21:52:57,962] - [remote_util:2995] INFO - running command.raw on 172.17.0.8: killall -SIGCONT memcached [2018-01-25 21:52:57,968] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:52:57,969] - [remote_util:2793] ERROR - bash: killall: command not found [2018-01-25 21:52:57,970] - [remote_util:2995] INFO - running command.raw on 172.17.0.8: killall -SIGCONT beam [2018-01-25 21:52:58,039] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:52:58,040] - [remote_util:2793] ERROR - bash: killall: command not found [2018-01-25 21:52:58,157] - [remote_util:232] INFO - connecting to 172.17.0.6 with username:root password:couchbase ssh_key: [2018-01-25 21:52:58,241] - [remote_util:266] INFO - Connected to 172.17.0.6 [2018-01-25 21:52:58,649] - [remote_util:2995] INFO - running command.raw on 172.17.0.6: /etc/init.d/couchbase-server start [2018-01-25 21:52:58,773] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:52:58,773] - [remote_util:2830] INFO - * couchbase-server is already started [2018-01-25 21:52:58,773] - [remote_util:2995] INFO - running command.raw on 172.17.0.6: /sbin/iptables -F [2018-01-25 21:52:58,779] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:52:58,779] - [remote_util:2793] ERROR - bash: /sbin/iptables: No such file or directory [2018-01-25 21:52:58,779] - [remote_util:2995] INFO - running command.raw on 172.17.0.6: /sbin/iptables -t nat -F [2018-01-25 21:52:58,850] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:52:58,851] - [remote_util:2793] ERROR - bash: /sbin/iptables: No such file or directory [2018-01-25 21:52:58,851] - [remote_util:291] INFO - Connect to node: 172.17.0.6 as user: root [2018-01-25 21:52:58,987] - [remote_util:314] INFO - Connected to 172.17.0.6 as root [2018-01-25 21:52:58,988] - [remote_util:3450] INFO - *** unpause memcached process *** [2018-01-25 21:52:58,988] - [remote_util:2995] INFO - running command.raw on 172.17.0.6: killall -SIGCONT memcached [2018-01-25 21:52:58,994] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:52:58,994] - [remote_util:2793] ERROR - bash: killall: command not found [2018-01-25 21:52:58,995] - [remote_util:2995] INFO - running command.raw on 172.17.0.6: killall -SIGCONT beam [2018-01-25 21:52:59,067] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:52:59,067] - [remote_util:2793] ERROR - bash: killall: command not found [2018-01-25 21:52:59,071] - [remote_util:232] INFO - connecting to 172.17.0.7 with username:root password:couchbase ssh_key: [2018-01-25 21:52:59,180] - [remote_util:266] INFO - Connected to 172.17.0.7 [2018-01-25 21:52:59,702] - [remote_util:2995] INFO - running command.raw on 172.17.0.7: /etc/init.d/couchbase-server start [2018-01-25 21:52:59,712] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:52:59,712] - [remote_util:2830] INFO - * couchbase-server is already started [2018-01-25 21:52:59,712] - [remote_util:2995] INFO - running command.raw on 172.17.0.7: /sbin/iptables -F [2018-01-25 21:52:59,783] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:52:59,783] - [remote_util:2793] ERROR - bash: /sbin/iptables: No such file or directory [2018-01-25 21:52:59,783] - [remote_util:2995] INFO - running command.raw on 172.17.0.7: /sbin/iptables -t nat -F [2018-01-25 21:52:59,854] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:52:59,854] - [remote_util:2793] ERROR - bash: /sbin/iptables: No such file or directory [2018-01-25 21:52:59,854] - [remote_util:291] INFO - Connect to node: 172.17.0.7 as user: root [2018-01-25 21:52:59,987] - [remote_util:314] INFO - Connected to 172.17.0.7 as root [2018-01-25 21:52:59,987] - [remote_util:3450] INFO - *** unpause memcached process *** [2018-01-25 21:52:59,987] - [remote_util:2995] INFO - running command.raw on 172.17.0.7: killall -SIGCONT memcached [2018-01-25 21:53:00,964] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:53:00,964] - [remote_util:2793] ERROR - bash: killall: command not found [2018-01-25 21:53:00,964] - [remote_util:2995] INFO - running command.raw on 172.17.0.7: killall -SIGCONT beam [2018-01-25 21:53:01,038] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:53:01,039] - [remote_util:2793] ERROR - bash: killall: command not found [2018-01-25 21:53:01,189] - [remote_util:232] INFO - connecting to 172.17.0.5 with username:root password:couchbase ssh_key: [2018-01-25 21:53:01,295] - [remote_util:266] INFO - Connected to 172.17.0.5 [2018-01-25 21:53:01,982] - [remote_util:2995] INFO - running command.raw on 172.17.0.5: /etc/init.d/couchbase-server start [2018-01-25 21:53:01,994] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:53:01,994] - [remote_util:2830] INFO - * couchbase-server is already started [2018-01-25 21:53:01,995] - [remote_util:2995] INFO - running command.raw on 172.17.0.5: /sbin/iptables -F [2018-01-25 21:53:02,070] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:53:02,072] - [remote_util:2793] ERROR - bash: /sbin/iptables: No such file or directory [2018-01-25 21:53:02,073] - [remote_util:2995] INFO - running command.raw on 172.17.0.5: /sbin/iptables -t nat -F [2018-01-25 21:53:02,143] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:53:02,143] - [remote_util:2793] ERROR - bash: /sbin/iptables: No such file or directory [2018-01-25 21:53:02,144] - [remote_util:291] INFO - Connect to node: 172.17.0.5 as user: root [2018-01-25 21:53:02,275] - [remote_util:314] INFO - Connected to 172.17.0.5 as root [2018-01-25 21:53:02,276] - [remote_util:3450] INFO - *** unpause memcached process *** [2018-01-25 21:53:02,276] - [remote_util:2995] INFO - running command.raw on 172.17.0.5: killall -SIGCONT memcached [2018-01-25 21:53:02,286] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:53:02,287] - [remote_util:2793] ERROR - bash: killall: command not found [2018-01-25 21:53:02,287] - [remote_util:2995] INFO - running command.raw on 172.17.0.5: killall -SIGCONT beam [2018-01-25 21:53:02,358] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:53:02,359] - [remote_util:2793] ERROR - bash: killall: command not found [2018-01-25 21:53:12,486] - [failoverbasetests:84] INFO - ============== tearDown was finished for test #18 test_failover_normal ============== [2018-01-25 21:53:12,487] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 21:53:12,655] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 21:53:13,099] - [remote_util:2995] INFO - running command.raw on 172.17.0.8: kill -9 $(ps aux | pgrep 'memcached') [2018-01-25 21:53:13,187] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:53:13,305] - [remote_util:232] INFO - connecting to 172.17.0.6 with username:root password:couchbase ssh_key: [2018-01-25 21:53:13,437] - [remote_util:266] INFO - Connected to 172.17.0.6 [2018-01-25 21:53:13,923] - [remote_util:2995] INFO - running command.raw on 172.17.0.6: kill -9 $(ps aux | pgrep 'memcached') [2018-01-25 21:53:14,005] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:53:14,009] - [remote_util:232] INFO - connecting to 172.17.0.7 with username:root password:couchbase ssh_key: [2018-01-25 21:53:14,181] - [remote_util:266] INFO - Connected to 172.17.0.7 [2018-01-25 21:53:14,609] - [remote_util:2995] INFO - running command.raw on 172.17.0.7: kill -9 $(ps aux | pgrep 'memcached') [2018-01-25 21:53:14,681] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:53:14,798] - [remote_util:232] INFO - connecting to 172.17.0.5 with username:root password:couchbase ssh_key: [2018-01-25 21:53:15,021] - [remote_util:266] INFO - Connected to 172.17.0.5 [2018-01-25 21:53:16,229] - [remote_util:2995] INFO - running command.raw on 172.17.0.5: kill -9 $(ps aux | pgrep 'memcached') [2018-01-25 21:53:16,312] - [remote_util:3032] INFO - command executed successfully ====================================================================== ERROR: test_failover_normal (failover.failovertests.FailoverTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "pytests/failover/failovertests.py", line 22, in test_failover_normal self.common_test_body('normal') File "pytests/failover/failovertests.py", line 75, in common_test_body self.load_initial_data() File "pytests/failover/failovertests.py", line 426, in load_initial_data self._verify_stats_all_buckets(self.servers, timeout=120) File "pytests/basetestcase.py", line 824, in _verify_stats_all_buckets raise Exception("unable to get expected stats during {0} sec".format(timeout)) Exception: unable to get expected stats during 120 sec ---------------------------------------------------------------------- Ran 1 test in 235.777s FAILED (errors=1) Cluster instance shutdown with force summary so far suite recreatebuckettests.RecreateMembaseBuckets , pass 0 , fail 1 failures so far... recreatebuckettests.RecreateMembaseBuckets.test_default_moxi summary so far suite deletebuckettests.DeleteMembaseBuckets , pass 1 , fail 0 summary so far suite createbuckettests.CreateMembaseBucketsTests , pass 1 , fail 4 failures so far... createbuckettests.CreateMembaseBucketsTests.test_default_moxi createbuckettests.CreateMembaseBucketsTests.test_default_on_non_default_port createbuckettests.CreateMembaseBucketsTests.test_non_default_case_sensitive_different_port createbuckettests.CreateMembaseBucketsTests.test_two_replica summary so far suite setgettests.MembaseBucket , pass 1 , fail 0 summary so far suite expirytests.ExpiryTests , pass 1 , fail 0 summary so far suite memcapable.GetlTests , pass 2 , fail 0 summary so far suite memorysanitytests.MemorySanity , pass 1 , fail 0 summary so far suite drainratetests.DrainRateTests , pass 1 , fail 0 summary so far suite view.viewquerytests.ViewQueryTests , pass 1 , fail 0 summary so far suite view.createdeleteview.CreateDeleteViewTests , pass 1 , fail 0 summary so far suite rebalance.rebalancein.RebalanceInTests , pass 0 , fail 1 failures so far... rebalance.rebalancein.RebalanceInTests.rebalance_in_with_ops summary so far suite rebalance.rebalanceout.RebalanceOutTests , pass 0 , fail 1 failures so far... rebalance.rebalanceout.RebalanceOutTests.rebalance_out_with_ops summary so far suite failover.failovertests.FailoverTests , pass 0 , fail 1 failures so far... failover.failovertests.FailoverTests.test_failover_normal testrunner logs, diags and results are available under /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_18 Logs will be stored at /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_19 ./testrunner -i ./4node.ini -t CCCP.CCCP.test_get_config_client,standard_buckets=1,sasl_buckets=1 Test Input params: {'cluster_name': '4node', 'logs_folder': '/testrunner/logs/testrunner-18-Jan-25_21-16-28/test_19', 'ini': './4node.ini', 'case_number': 19, 'sasl_buckets': '1', 'standard_buckets': '1', 'spec': '4node', 'conf_file': './4node.conf', 'num_nodes': 4} [2018-01-25 21:53:17,157] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 21:53:17,268] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 21:53:17,673] - [rest_client:1964] INFO - Node version in cluster 5.0.1-5003-enterprise [2018-01-25 21:53:17,681] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:53:17,681] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:53:17,681] - [basetestcase:201] INFO - ============== basetestcase setup was started for test #19 test_get_config_client============== [2018-01-25 21:53:17,707] - [bucket_helper:142] INFO - deleting existing buckets [u'default'] on 172.17.0.8 [2018-01-25 21:53:17,708] - [bucket_helper:144] INFO - remove bucket default ... [2018-01-25 21:53:19,760] - [bucket_helper:158] INFO - deleted bucket : default from 172.17.0.8 [2018-01-25 21:53:19,762] - [bucket_helper:236] INFO - waiting for bucket deletion to complete.... [2018-01-25 21:53:19,769] - [rest_client:133] INFO - node 172.17.0.8 existing buckets : [] [2018-01-25 21:53:19,769] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:53:21,780] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.6 [2018-01-25 21:53:21,780] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:53:23,791] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.7 [2018-01-25 21:53:23,791] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:53:25,951] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.5 [2018-01-25 21:53:25,952] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:53:27,979] - [cluster_helper:254] INFO - rebalancing all nodes in order to remove nodes [2018-01-25 21:53:27,984] - [rest_client:1423] INFO - rebalance params : {'password': 'password', 'ejectedNodes': u'ns_1@172.17.0.7,ns_1@172.17.0.5,ns_1@172.17.0.6', 'user': 'Administrator', 'knownNodes': u'ns_1@cb-example-0003.cb-example.default.svc,ns_1@172.17.0.7,ns_1@172.17.0.5,ns_1@172.17.0.6'} [2018-01-25 21:53:27,988] - [rest_client:807] ERROR - POST http://172.17.0.8:8091/controller/rebalance body: password=password&ejectedNodes=ns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6&user=Administrator&knownNodes=ns_1%40cb-example-0003.cb-example.default.svc%2Cns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6 headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 400 reason: status: 400, content: No active nodes left No active nodes left auth: Administrator:password [2018-01-25 21:53:27,990] - [rest_client:1430] ERROR - rebalance operation failed: No active nodes left test_get_config_client (CCCP.CCCP) ... Traceback (most recent call last): File "pytests/basetestcase.py", line 203, in setUp self._cluster_cleanup() File "pytests/basetestcase.py", line 494, in _cluster_cleanup ClusterOperationHelper.cleanup_cluster(self.servers, master=self.master) File "lib/membase/helper/cluster_helper.py", line 259, in cleanup_cluster wait_for_rebalance=wait_for_rebalance) File "lib/membase/api/rest_client.py", line 110, in remove_nodes self.rest.rebalance(knownNodes, ejectedNodes) File "lib/membase/api/rest_client.py", line 1433, in rebalance parameters=params) InvalidArgumentException: controller/rebalance with error message No active nodes left failed when invoked with parameters: password=password&ejectedNodes=ns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6&user=Administrator&knownNodes=ns_1%40cb-example-0003.cb-example.default.svc%2Cns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6 ERROR ====================================================================== ERROR: test_get_config_client (CCCP.CCCP) ---------------------------------------------------------------------- Traceback (most recent call last): File "pytests/CCCP.py", line 14, in setUp super(CCCP, self).setUp() File "pytests/basetestcase.py", line 352, in setUp self.fail(e) AssertionError: controller/rebalance with error message No active nodes left failed when invoked with parameters: password=password&ejectedNodes=ns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6&user=Administrator&knownNodes=ns_1%40cb-example-0003.cb-example.default.svc%2Cns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6 ---------------------------------------------------------------------- Ran 1 test in 10.836s FAILED (errors=1) Cluster instance shutdown with force summary so far suite recreatebuckettests.RecreateMembaseBuckets , pass 0 , fail 1 failures so far... recreatebuckettests.RecreateMembaseBuckets.test_default_moxi summary so far suite deletebuckettests.DeleteMembaseBuckets , pass 1 , fail 0 summary so far suite createbuckettests.CreateMembaseBucketsTests , pass 1 , fail 4 failures so far... createbuckettests.CreateMembaseBucketsTests.test_default_moxi createbuckettests.CreateMembaseBucketsTests.test_default_on_non_default_port createbuckettests.CreateMembaseBucketsTests.test_non_default_case_sensitive_different_port createbuckettests.CreateMembaseBucketsTests.test_two_replica summary so far suite setgettests.MembaseBucket , pass 1 , fail 0 summary so far suite expirytests.ExpiryTests , pass 1 , fail 0 summary so far suite memcapable.GetlTests , pass 2 , fail 0 summary so far suite memorysanitytests.MemorySanity , pass 1 , fail 0 summary so far suite drainratetests.DrainRateTests , pass 1 , fail 0 summary so far suite view.viewquerytests.ViewQueryTests , pass 1 , fail 0 summary so far suite view.createdeleteview.CreateDeleteViewTests , pass 1 , fail 0 summary so far suite rebalance.rebalancein.RebalanceInTests , pass 0 , fail 1 failures so far... rebalance.rebalancein.RebalanceInTests.rebalance_in_with_ops summary so far suite rebalance.rebalanceout.RebalanceOutTests , pass 0 , fail 1 failures so far... rebalance.rebalanceout.RebalanceOutTests.rebalance_out_with_ops summary so far suite failover.failovertests.FailoverTests , pass 0 , fail 1 failures so far... failover.failovertests.FailoverTests.test_failover_normal summary so far suite CCCP.CCCP , pass 0 , fail 1 failures so far... CCCP.CCCP.test_get_config_client testrunner logs, diags and results are available under /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_19 Logs will be stored at /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_20 ./testrunner -i ./4node.ini -t CCCP.CCCP.test_not_my_vbucket_config Test Input params: {'conf_file': './4node.conf', 'num_nodes': 4, 'cluster_name': '4node', 'ini': './4node.ini', 'case_number': 20, 'logs_folder': '/testrunner/logs/testrunner-18-Jan-25_21-16-28/test_20', 'spec': '4node'} [2018-01-25 21:53:28,002] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 21:53:28,133] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 21:53:28,754] - [rest_client:1964] INFO - Node version in cluster 5.0.1-5003-enterprise [2018-01-25 21:53:28,764] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:53:28,764] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:53:28,764] - [basetestcase:201] INFO - ============== basetestcase setup was started for test #20 test_not_my_vbucket_config============== [2018-01-25 21:53:28,780] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.8 [2018-01-25 21:53:28,780] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:53:30,942] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.6 [2018-01-25 21:53:30,942] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:53:32,953] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.7 [2018-01-25 21:53:32,953] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:53:34,964] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.5 [2018-01-25 21:53:34,965] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:53:36,993] - [cluster_helper:254] INFO - rebalancing all nodes in order to remove nodes [2018-01-25 21:53:36,996] - [rest_client:1423] INFO - rebalance params : {'password': 'password', 'ejectedNodes': u'ns_1@172.17.0.7,ns_1@172.17.0.5,ns_1@172.17.0.6', 'user': 'Administrator', 'knownNodes': u'ns_1@cb-example-0003.cb-example.default.svc,ns_1@172.17.0.7,ns_1@172.17.0.5,ns_1@172.17.0.6'} test_not_my_vbucket_config (CCCP.CCCP) ... Traceback (most recent call last): File "pytests/basetestcase.py", line 203, in setUp self._cluster_cleanup() File "pytests/basetestcase.py", line 494, in _cluster_cleanup ClusterOperationHelper.cleanup_cluster(self.servers, master=self.master) File "lib/membase/helper/cluster_helper.py", line 259, in cleanup_cluster wait_for_rebalance=wait_for_rebalance) File "lib/membase/api/rest_client.py", line 110, in remove_nodes self.rest.rebalance(knownNodes, ejectedNodes) File "lib/membase/api/rest_client.py", line 1433, in rebalance parameters=params) InvalidArgumentException: controller/rebalance with error message No active nodes left failed when invoked with parameters: password=password&ejectedNodes=ns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6&user=Administrator&knownNodes=ns_1%40cb-example-0003.cb-example.default.svc%2Cns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6 ERROR ====================================================================== ERROR: test_not_my_vbucket_config (CCCP.CCCP) ---------------------------------------------------------------------- Traceback (most recent call last): File "pytests/CCCP.py", line 14, in setUp super(CCCP, self).setUp() File "pytests/basetestcase.py", line 352, in setUp self.fail(e) AssertionError: controller/rebalance with error message No active nodes left failed when invoked with parameters: password=password&ejectedNodes=ns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6&user=Administrator&knownNodes=ns_1%40cb-example-0003.cb-example.default.svc%2Cns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6 ---------------------------------------------------------------------- Ran 1 test in 9.023s FAILED (errors=1) [2018-01-25 21:53:37,022] - [rest_client:807] ERROR - POST http://172.17.0.8:8091/controller/rebalance body: password=password&ejectedNodes=ns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6&user=Administrator&knownNodes=ns_1%40cb-example-0003.cb-example.default.svc%2Cns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6 headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 400 reason: status: 400, content: No active nodes left No active nodes left auth: Administrator:password [2018-01-25 21:53:37,022] - [rest_client:1430] ERROR - rebalance operation failed: No active nodes left Cluster instance shutdown with force summary so far suite recreatebuckettests.RecreateMembaseBuckets , pass 0 , fail 1 failures so far... recreatebuckettests.RecreateMembaseBuckets.test_default_moxi summary so far suite deletebuckettests.DeleteMembaseBuckets , pass 1 , fail 0 summary so far suite createbuckettests.CreateMembaseBucketsTests , pass 1 , fail 4 failures so far... createbuckettests.CreateMembaseBucketsTests.test_default_moxi createbuckettests.CreateMembaseBucketsTests.test_default_on_non_default_port createbuckettests.CreateMembaseBucketsTests.test_non_default_case_sensitive_different_port createbuckettests.CreateMembaseBucketsTests.test_two_replica summary so far suite setgettests.MembaseBucket , pass 1 , fail 0 summary so far suite expirytests.ExpiryTests , pass 1 , fail 0 summary so far suite memcapable.GetlTests , pass 2 , fail 0 summary so far suite memorysanitytests.MemorySanity , pass 1 , fail 0 summary so far suite drainratetests.DrainRateTests , pass 1 , fail 0 summary so far suite view.viewquerytests.ViewQueryTests , pass 1 , fail 0 summary so far suite view.createdeleteview.CreateDeleteViewTests , pass 1 , fail 0 summary so far suite rebalance.rebalancein.RebalanceInTests , pass 0 , fail 1 failures so far... rebalance.rebalancein.RebalanceInTests.rebalance_in_with_ops summary so far suite rebalance.rebalanceout.RebalanceOutTests , pass 0 , fail 1 failures so far... rebalance.rebalanceout.RebalanceOutTests.rebalance_out_with_ops summary so far suite failover.failovertests.FailoverTests , pass 0 , fail 1 failures so far... failover.failovertests.FailoverTests.test_failover_normal summary so far suite CCCP.CCCP , pass 0 , fail 2 failures so far... CCCP.CCCP.test_get_config_client CCCP.CCCP.test_not_my_vbucket_config testrunner logs, diags and results are available under /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_20 Logs will be stored at /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_21 ./testrunner -i ./4node.ini -t flush.bucketflush.BucketFlushTests.bucketflush_with_data_ops_moxi,items=5000,data_op=create,use_ascii=False Test Input params: {'items': '5000', 'case_number': 21, 'conf_file': './4node.conf', 'num_nodes': 4, 'cluster_name': '4node', 'use_ascii': 'False', 'ini': './4node.ini', 'data_op': 'create', 'logs_folder': '/testrunner/logs/testrunner-18-Jan-25_21-16-28/test_21', 'spec': '4node'} [2018-01-25 21:53:37,047] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 21:53:37,187] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 21:53:38,154] - [rest_client:1964] INFO - Node version in cluster 5.0.1-5003-enterprise [2018-01-25 21:53:38,176] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:53:38,177] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:53:38,177] - [basetestcase:201] INFO - ============== basetestcase setup was started for test #21 bucketflush_with_data_ops_moxi============== [2018-01-25 21:53:38,220] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.8 [2018-01-25 21:53:38,220] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:53:40,236] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.6 [2018-01-25 21:53:40,236] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:53:42,248] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.7 [2018-01-25 21:53:42,248] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:53:44,257] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.5 [2018-01-25 21:53:44,258] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:53:46,276] - [cluster_helper:254] INFO - rebalancing all nodes in order to remove nodes [2018-01-25 21:53:46,279] - [rest_client:1423] INFO - rebalance params : {'password': 'password', 'ejectedNodes': u'ns_1@172.17.0.7,ns_1@172.17.0.5,ns_1@172.17.0.6', 'user': 'Administrator', 'knownNodes': u'ns_1@cb-example-0003.cb-example.default.svc,ns_1@172.17.0.7,ns_1@172.17.0.5,ns_1@172.17.0.6'} [2018-01-25 21:53:46,282] - [rest_client:807] ERROR - POST http://172.17.0.8:8091/controller/rebalance body: password=password&ejectedNodes=ns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6&user=Administrator&knownNodes=ns_1%40cb-example-0003.cb-example.default.svc%2Cns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6 headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 400 reason: status: 400, content: No active nodes left No active nodes left auth: Administrator:password [2018-01-25 21:53:46,283] - [rest_client:1430] ERROR - rebalance operation failed: No active nodes left bucketflush_with_data_ops_moxi (flush.bucketflush.BucketFlushTests) ... Traceback (most recent call last): File "pytests/basetestcase.py", line 203, in setUp self._cluster_cleanup() File "pytests/basetestcase.py", line 494, in _cluster_cleanup ClusterOperationHelper.cleanup_cluster(self.servers, master=self.master) File "lib/membase/helper/cluster_helper.py", line 259, in cleanup_cluster wait_for_rebalance=wait_for_rebalance) File "lib/membase/api/rest_client.py", line 110, in remove_nodes self.rest.rebalance(knownNodes, ejectedNodes) File "lib/membase/api/rest_client.py", line 1433, in rebalance parameters=params) InvalidArgumentException: controller/rebalance with error message No active nodes left failed when invoked with parameters: password=password&ejectedNodes=ns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6&user=Administrator&knownNodes=ns_1%40cb-example-0003.cb-example.default.svc%2Cns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6 ERROR ====================================================================== ERROR: bucketflush_with_data_ops_moxi (flush.bucketflush.BucketFlushTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "pytests/flush/bucketflush.py", line 12, in setUp super(BucketFlushTests, self).setUp() File "pytests/basetestcase.py", line 352, in setUp self.fail(e) AssertionError: controller/rebalance with error message No active nodes left failed when invoked with parameters: password=password&ejectedNodes=ns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6&user=Administrator&knownNodes=ns_1%40cb-example-0003.cb-example.default.svc%2Cns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6 ---------------------------------------------------------------------- Ran 1 test in 9.239s FAILED (errors=1) Cluster instance shutdown with force summary so far suite recreatebuckettests.RecreateMembaseBuckets , pass 0 , fail 1 failures so far... recreatebuckettests.RecreateMembaseBuckets.test_default_moxi summary so far suite deletebuckettests.DeleteMembaseBuckets , pass 1 , fail 0 summary so far suite createbuckettests.CreateMembaseBucketsTests , pass 1 , fail 4 failures so far... createbuckettests.CreateMembaseBucketsTests.test_default_moxi createbuckettests.CreateMembaseBucketsTests.test_default_on_non_default_port createbuckettests.CreateMembaseBucketsTests.test_non_default_case_sensitive_different_port createbuckettests.CreateMembaseBucketsTests.test_two_replica summary so far suite setgettests.MembaseBucket , pass 1 , fail 0 summary so far suite expirytests.ExpiryTests , pass 1 , fail 0 summary so far suite memcapable.GetlTests , pass 2 , fail 0 summary so far suite memorysanitytests.MemorySanity , pass 1 , fail 0 summary so far suite drainratetests.DrainRateTests , pass 1 , fail 0 summary so far suite view.viewquerytests.ViewQueryTests , pass 1 , fail 0 summary so far suite view.createdeleteview.CreateDeleteViewTests , pass 1 , fail 0 summary so far suite rebalance.rebalancein.RebalanceInTests , pass 0 , fail 1 failures so far... rebalance.rebalancein.RebalanceInTests.rebalance_in_with_ops summary so far suite rebalance.rebalanceout.RebalanceOutTests , pass 0 , fail 1 failures so far... rebalance.rebalanceout.RebalanceOutTests.rebalance_out_with_ops summary so far suite failover.failovertests.FailoverTests , pass 0 , fail 1 failures so far... failover.failovertests.FailoverTests.test_failover_normal summary so far suite CCCP.CCCP , pass 0 , fail 2 failures so far... CCCP.CCCP.test_get_config_client CCCP.CCCP.test_not_my_vbucket_config summary so far suite flush.bucketflush.BucketFlushTests , pass 0 , fail 1 failures so far... flush.bucketflush.BucketFlushTests.bucketflush_with_data_ops_moxi testrunner logs, diags and results are available under /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_21 Logs will be stored at /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_22 ./testrunner -i ./4node.ini -t security.audittest.auditTest.test_bucketEvents,default_bucket=false,id=8201,ops=create Test Input params: {'ops': 'create', 'spec': '4node', 'conf_file': './4node.conf', 'num_nodes': 4, 'cluster_name': '4node', 'ini': './4node.ini', 'case_number': 22, 'logs_folder': '/testrunner/logs/testrunner-18-Jan-25_21-16-28/test_22', 'id': '8201', 'default_bucket': 'false'} [2018-01-25 21:53:46,326] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 21:53:46,458] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 21:53:46,861] - [rest_client:1964] INFO - Node version in cluster 5.0.1-5003-enterprise [2018-01-25 21:53:46,873] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:53:46,873] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:53:46,873] - [basetestcase:201] INFO - ============== basetestcase setup was started for test #22 test_bucketEvents============== [2018-01-25 21:53:46,899] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.8 [2018-01-25 21:53:46,974] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:53:48,986] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.6 [2018-01-25 21:53:48,986] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:53:51,004] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.7 [2018-01-25 21:53:51,005] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:53:53,018] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.5 [2018-01-25 21:53:53,018] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:53:55,053] - [cluster_helper:254] INFO - rebalancing all nodes in order to remove nodes [2018-01-25 21:53:55,057] - [rest_client:1423] INFO - rebalance params : {'password': 'password', 'ejectedNodes': u'ns_1@172.17.0.7,ns_1@172.17.0.5,ns_1@172.17.0.6', 'user': 'Administrator', 'knownNodes': u'ns_1@cb-example-0003.cb-example.default.svc,ns_1@172.17.0.7,ns_1@172.17.0.5,ns_1@172.17.0.6'} test_bucketEvents (security.audittest.auditTest) ... Traceback (most recent call last): File "pytests/basetestcase.py", line 203, in setUp self._cluster_cleanup() File "pytests/basetestcase.py", line 494, in _cluster_cleanup ClusterOperationHelper.cleanup_cluster(self.servers, master=self.master) File "lib/membase/helper/cluster_helper.py", line 259, in cleanup_cluster wait_for_rebalance=wait_for_rebalance) File "lib/membase/api/rest_client.py", line 110, in remove_nodes self.rest.rebalance(knownNodes, ejectedNodes) File "lib/membase/api/rest_client.py", line 1433, in rebalance parameters=params) InvalidArgumentException: controller/rebalance with error message No active nodes left failed when invoked with parameters: password=password&ejectedNodes=ns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6&user=Administrator&knownNodes=ns_1%40cb-example-0003.cb-example.default.svc%2Cns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6 ERROR ====================================================================== ERROR: test_bucketEvents (security.audittest.auditTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "pytests/security/audittest.py", line 32, in setUp super(auditTest, self).setUp() File "pytests/basetestcase.py", line 352, in setUp self.fail(e) AssertionError: controller/rebalance with error message No active nodes left failed when invoked with parameters: password=password&ejectedNodes=ns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6&user=Administrator&knownNodes=ns_1%40cb-example-0003.cb-example.default.svc%2Cns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6 ---------------------------------------------------------------------- Ran 1 test in 8.739s FAILED (errors=1) [2018-01-25 21:53:55,063] - [rest_client:807] ERROR - POST http://172.17.0.8:8091/controller/rebalance body: password=password&ejectedNodes=ns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6&user=Administrator&knownNodes=ns_1%40cb-example-0003.cb-example.default.svc%2Cns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6 headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 400 reason: status: 400, content: No active nodes left No active nodes left auth: Administrator:password [2018-01-25 21:53:55,064] - [rest_client:1430] ERROR - rebalance operation failed: No active nodes left Cluster instance shutdown with force summary so far suite recreatebuckettests.RecreateMembaseBuckets , pass 0 , fail 1 failures so far... recreatebuckettests.RecreateMembaseBuckets.test_default_moxi summary so far suite deletebuckettests.DeleteMembaseBuckets , pass 1 , fail 0 summary so far suite createbuckettests.CreateMembaseBucketsTests , pass 1 , fail 4 failures so far... createbuckettests.CreateMembaseBucketsTests.test_default_moxi createbuckettests.CreateMembaseBucketsTests.test_default_on_non_default_port createbuckettests.CreateMembaseBucketsTests.test_non_default_case_sensitive_different_port createbuckettests.CreateMembaseBucketsTests.test_two_replica summary so far suite setgettests.MembaseBucket , pass 1 , fail 0 summary so far suite expirytests.ExpiryTests , pass 1 , fail 0 summary so far suite memcapable.GetlTests , pass 2 , fail 0 summary so far suite memorysanitytests.MemorySanity , pass 1 , fail 0 summary so far suite drainratetests.DrainRateTests , pass 1 , fail 0 summary so far suite view.viewquerytests.ViewQueryTests , pass 1 , fail 0 summary so far suite view.createdeleteview.CreateDeleteViewTests , pass 1 , fail 0 summary so far suite rebalance.rebalancein.RebalanceInTests , pass 0 , fail 1 failures so far... rebalance.rebalancein.RebalanceInTests.rebalance_in_with_ops summary so far suite rebalance.rebalanceout.RebalanceOutTests , pass 0 , fail 1 failures so far... rebalance.rebalanceout.RebalanceOutTests.rebalance_out_with_ops summary so far suite failover.failovertests.FailoverTests , pass 0 , fail 1 failures so far... failover.failovertests.FailoverTests.test_failover_normal summary so far suite CCCP.CCCP , pass 0 , fail 2 failures so far... CCCP.CCCP.test_get_config_client CCCP.CCCP.test_not_my_vbucket_config summary so far suite flush.bucketflush.BucketFlushTests , pass 0 , fail 1 failures so far... flush.bucketflush.BucketFlushTests.bucketflush_with_data_ops_moxi summary so far suite security.audittest.auditTest , pass 0 , fail 1 failures so far... security.audittest.auditTest.test_bucketEvents testrunner logs, diags and results are available under /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_22 Logs will be stored at /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_23 ./testrunner -i ./4node.ini -t tuqquery.tuq_precedence.PrecedenceTests.test_case_and_like,primary_indx_type=GSI,doc-per-day=1,force_clean=True,reload_data=True Test Input params: {'doc-per-day': '1', 'primary_indx_type': 'GSI', 'conf_file': './4node.conf', 'num_nodes': 4, 'cluster_name': '4node', 'reload_data': 'True', 'force_clean': 'True', 'ini': './4node.ini', 'case_number': 23, 'logs_folder': '/testrunner/logs/testrunner-18-Jan-25_21-16-28/test_23', 'spec': '4node'} [2018-01-25 21:53:55,139] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 21:53:55,272] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 21:53:55,698] - [rest_client:1964] INFO - Node version in cluster 5.0.1-5003-enterprise [2018-01-25 21:53:55,708] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:53:55,708] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:53:55,709] - [basetestcase:201] INFO - ============== basetestcase setup was started for test #23 test_case_and_like============== [2018-01-25 21:53:55,709] - [basetestcase:230] INFO - any cluster operation in setup will be skipped [2018-01-25 21:53:55,709] - [basetestcase:233] INFO - ============== basetestcase setup was finished for test #23 test_case_and_like ============== [2018-01-25 21:53:55,710] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 21:53:55,842] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 21:53:56,300] - [rest_client:1964] INFO - Node version in cluster 5.0.1-5003-enterprise [2018-01-25 21:53:56,307] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:53:56,307] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:53:56,307] - [basetestcase:201] INFO - ============== basetestcase setup was started for test #23 test_case_and_like============== [2018-01-25 21:53:56,322] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.8 [2018-01-25 21:53:56,323] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:53:58,335] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.6 [2018-01-25 21:53:58,336] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:54:00,343] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.7 [2018-01-25 21:54:00,343] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:54:02,360] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.5 [2018-01-25 21:54:02,361] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:54:04,379] - [cluster_helper:254] INFO - rebalancing all nodes in order to remove nodes [2018-01-25 21:54:04,381] - [rest_client:1423] INFO - rebalance params : {'password': 'password', 'ejectedNodes': u'ns_1@172.17.0.7,ns_1@172.17.0.5,ns_1@172.17.0.6', 'user': 'Administrator', 'knownNodes': u'ns_1@cb-example-0003.cb-example.default.svc,ns_1@172.17.0.7,ns_1@172.17.0.5,ns_1@172.17.0.6'} [2018-01-25 21:54:04,383] - [rest_client:807] ERROR - POST http://172.17.0.8:8091/controller/rebalance body: password=password&ejectedNodes=ns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6&user=Administrator&knownNodes=ns_1%40cb-example-0003.cb-example.default.svc%2Cns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6 headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 400 reason: status: 400, content: No active nodes left No active nodes left auth: Administrator:password [2018-01-25 21:54:04,385] - [rest_client:1430] ERROR - rebalance operation failed: No active nodes left test_case_and_like (tuqquery.tuq_precedence.PrecedenceTests) ... Traceback (most recent call last): File "pytests/basetestcase.py", line 203, in setUp self._cluster_cleanup() File "pytests/basetestcase.py", line 494, in _cluster_cleanup ClusterOperationHelper.cleanup_cluster(self.servers, master=self.master) File "lib/membase/helper/cluster_helper.py", line 259, in cleanup_cluster wait_for_rebalance=wait_for_rebalance) File "lib/membase/api/rest_client.py", line 110, in remove_nodes self.rest.rebalance(knownNodes, ejectedNodes) File "lib/membase/api/rest_client.py", line 1433, in rebalance parameters=params) InvalidArgumentException: controller/rebalance with error message No active nodes left failed when invoked with parameters: password=password&ejectedNodes=ns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6&user=Administrator&knownNodes=ns_1%40cb-example-0003.cb-example.default.svc%2Cns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6 ERROR ====================================================================== ERROR: test_case_and_like (tuqquery.tuq_precedence.PrecedenceTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "pytests/tuqquery/tuq_precedence.py", line 7, in setUp super(PrecedenceTests, self).setUp() File "pytests/tuqquery/tuq.py", line 40, in setUp super(QueryTests, self).setUp() File "pytests/basetestcase.py", line 352, in setUp self.fail(e) AssertionError: controller/rebalance with error message No active nodes left failed when invoked with parameters: password=password&ejectedNodes=ns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6&user=Administrator&knownNodes=ns_1%40cb-example-0003.cb-example.default.svc%2Cns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6 ---------------------------------------------------------------------- Ran 1 test in 9.248s FAILED (errors=1) Cluster instance shutdown with force summary so far suite recreatebuckettests.RecreateMembaseBuckets , pass 0 , fail 1 failures so far... recreatebuckettests.RecreateMembaseBuckets.test_default_moxi summary so far suite deletebuckettests.DeleteMembaseBuckets , pass 1 , fail 0 summary so far suite createbuckettests.CreateMembaseBucketsTests , pass 1 , fail 4 failures so far... createbuckettests.CreateMembaseBucketsTests.test_default_moxi createbuckettests.CreateMembaseBucketsTests.test_default_on_non_default_port createbuckettests.CreateMembaseBucketsTests.test_non_default_case_sensitive_different_port createbuckettests.CreateMembaseBucketsTests.test_two_replica summary so far suite setgettests.MembaseBucket , pass 1 , fail 0 summary so far suite expirytests.ExpiryTests , pass 1 , fail 0 summary so far suite memcapable.GetlTests , pass 2 , fail 0 summary so far suite memorysanitytests.MemorySanity , pass 1 , fail 0 summary so far suite drainratetests.DrainRateTests , pass 1 , fail 0 summary so far suite view.viewquerytests.ViewQueryTests , pass 1 , fail 0 summary so far suite view.createdeleteview.CreateDeleteViewTests , pass 1 , fail 0 summary so far suite rebalance.rebalancein.RebalanceInTests , pass 0 , fail 1 failures so far... rebalance.rebalancein.RebalanceInTests.rebalance_in_with_ops summary so far suite rebalance.rebalanceout.RebalanceOutTests , pass 0 , fail 1 failures so far... rebalance.rebalanceout.RebalanceOutTests.rebalance_out_with_ops summary so far suite failover.failovertests.FailoverTests , pass 0 , fail 1 failures so far... failover.failovertests.FailoverTests.test_failover_normal summary so far suite CCCP.CCCP , pass 0 , fail 2 failures so far... CCCP.CCCP.test_get_config_client CCCP.CCCP.test_not_my_vbucket_config summary so far suite flush.bucketflush.BucketFlushTests , pass 0 , fail 1 failures so far... flush.bucketflush.BucketFlushTests.bucketflush_with_data_ops_moxi summary so far suite security.audittest.auditTest , pass 0 , fail 1 failures so far... security.audittest.auditTest.test_bucketEvents summary so far suite tuqquery.tuq_precedence.PrecedenceTests , pass 0 , fail 1 failures so far... tuqquery.tuq_precedence.PrecedenceTests.test_case_and_like testrunner logs, diags and results are available under /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_23 Logs will be stored at /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_24 ./testrunner -i ./4node.ini -t tuqquery.tuq_precedence.PrecedenceTests.test_case_and_like,doc-per-day=1,force_clean=True,reload_data=True Test Input params: {'doc-per-day': '1', 'conf_file': './4node.conf', 'num_nodes': 4, 'cluster_name': '4node', 'reload_data': 'True', 'force_clean': 'True', 'ini': './4node.ini', 'case_number': 24, 'logs_folder': '/testrunner/logs/testrunner-18-Jan-25_21-16-28/test_24', 'spec': '4node'} [2018-01-25 21:54:04,412] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 21:54:04,493] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 21:54:04,919] - [rest_client:1964] INFO - Node version in cluster 5.0.1-5003-enterprise [2018-01-25 21:54:04,926] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:54:04,927] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:54:04,927] - [basetestcase:201] INFO - ============== basetestcase setup was started for test #24 test_case_and_like============== [2018-01-25 21:54:04,927] - [basetestcase:230] INFO - any cluster operation in setup will be skipped [2018-01-25 21:54:04,927] - [basetestcase:233] INFO - ============== basetestcase setup was finished for test #24 test_case_and_like ============== [2018-01-25 21:54:04,928] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 21:54:05,059] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 21:54:05,457] - [rest_client:1964] INFO - Node version in cluster 5.0.1-5003-enterprise [2018-01-25 21:54:05,464] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:54:05,464] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:54:05,465] - [basetestcase:201] INFO - ============== basetestcase setup was started for test #24 test_case_and_like============== [2018-01-25 21:54:05,495] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.8 [2018-01-25 21:54:05,495] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:54:07,514] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.6 [2018-01-25 21:54:07,515] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:54:09,539] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.7 [2018-01-25 21:54:09,539] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:54:11,551] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.5 [2018-01-25 21:54:11,552] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:54:13,571] - [cluster_helper:254] INFO - rebalancing all nodes in order to remove nodes test_case_and_like (tuqquery.tuq_precedence.PrecedenceTests) ... Traceback (most recent call last): File "pytests/basetestcase.py", line 203, in setUp self._cluster_cleanup() File "pytests/basetestcase.py", line 494, in _cluster_cleanup ClusterOperationHelper.cleanup_cluster(self.servers, master=self.master) File "lib/membase/helper/cluster_helper.py", line 259, in cleanup_cluster wait_for_rebalance=wait_for_rebalance) File "lib/membase/api/rest_client.py", line 110, in remove_nodes self.rest.rebalance(knownNodes, ejectedNodes) File "lib/membase/api/rest_client.py", line 1433, in rebalance parameters=params) InvalidArgumentException: controller/rebalance with error message No active nodes left failed when invoked with parameters: password=password&ejectedNodes=ns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6&user=Administrator&knownNodes=ns_1%40cb-example-0003.cb-example.default.svc%2Cns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6 ERROR ====================================================================== ERROR: test_case_and_like (tuqquery.tuq_precedence.PrecedenceTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "pytests/tuqquery/tuq_precedence.py", line 7, in setUp super(PrecedenceTests, self).setUp() File "pytests/tuqquery/tuq.py", line 40, in setUp super(QueryTests, self).setUp() File "pytests/basetestcase.py", line 352, in setUp self.fail(e) AssertionError: controller/rebalance with error message No active nodes left failed when invoked with parameters: password=password&ejectedNodes=ns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6&user=Administrator&knownNodes=ns_1%40cb-example-0003.cb-example.default.svc%2Cns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6 ---------------------------------------------------------------------- Ran 1 test in 9.170s FAILED (errors=1) [2018-01-25 21:54:13,577] - [rest_client:1423] INFO - rebalance params : {'password': 'password', 'ejectedNodes': u'ns_1@172.17.0.7,ns_1@172.17.0.5,ns_1@172.17.0.6', 'user': 'Administrator', 'knownNodes': u'ns_1@cb-example-0003.cb-example.default.svc,ns_1@172.17.0.7,ns_1@172.17.0.5,ns_1@172.17.0.6'} [2018-01-25 21:54:13,580] - [rest_client:807] ERROR - POST http://172.17.0.8:8091/controller/rebalance body: password=password&ejectedNodes=ns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6&user=Administrator&knownNodes=ns_1%40cb-example-0003.cb-example.default.svc%2Cns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6 headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 400 reason: status: 400, content: No active nodes left No active nodes left auth: Administrator:password [2018-01-25 21:54:13,580] - [rest_client:1430] ERROR - rebalance operation failed: No active nodes left Cluster instance shutdown with force summary so far suite recreatebuckettests.RecreateMembaseBuckets , pass 0 , fail 1 failures so far... recreatebuckettests.RecreateMembaseBuckets.test_default_moxi summary so far suite deletebuckettests.DeleteMembaseBuckets , pass 1 , fail 0 summary so far suite createbuckettests.CreateMembaseBucketsTests , pass 1 , fail 4 failures so far... createbuckettests.CreateMembaseBucketsTests.test_default_moxi createbuckettests.CreateMembaseBucketsTests.test_default_on_non_default_port createbuckettests.CreateMembaseBucketsTests.test_non_default_case_sensitive_different_port createbuckettests.CreateMembaseBucketsTests.test_two_replica summary so far suite setgettests.MembaseBucket , pass 1 , fail 0 summary so far suite expirytests.ExpiryTests , pass 1 , fail 0 summary so far suite memcapable.GetlTests , pass 2 , fail 0 summary so far suite memorysanitytests.MemorySanity , pass 1 , fail 0 summary so far suite drainratetests.DrainRateTests , pass 1 , fail 0 summary so far suite view.viewquerytests.ViewQueryTests , pass 1 , fail 0 summary so far suite view.createdeleteview.CreateDeleteViewTests , pass 1 , fail 0 summary so far suite rebalance.rebalancein.RebalanceInTests , pass 0 , fail 1 failures so far... rebalance.rebalancein.RebalanceInTests.rebalance_in_with_ops summary so far suite rebalance.rebalanceout.RebalanceOutTests , pass 0 , fail 1 failures so far... rebalance.rebalanceout.RebalanceOutTests.rebalance_out_with_ops summary so far suite failover.failovertests.FailoverTests , pass 0 , fail 1 failures so far... failover.failovertests.FailoverTests.test_failover_normal summary so far suite CCCP.CCCP , pass 0 , fail 2 failures so far... CCCP.CCCP.test_get_config_client CCCP.CCCP.test_not_my_vbucket_config summary so far suite flush.bucketflush.BucketFlushTests , pass 0 , fail 1 failures so far... flush.bucketflush.BucketFlushTests.bucketflush_with_data_ops_moxi summary so far suite security.audittest.auditTest , pass 0 , fail 1 failures so far... security.audittest.auditTest.test_bucketEvents summary so far suite tuqquery.tuq_precedence.PrecedenceTests , pass 0 , fail 2 failures so far... tuqquery.tuq_precedence.PrecedenceTests.test_case_and_like tuqquery.tuq_precedence.PrecedenceTests.test_case_and_like testrunner logs, diags and results are available under /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_24 Logs will be stored at /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_25 ./testrunner -i ./4node.ini -t tuqquery.tuq_index.QueriesViewsTests.test_primary_create_delete_index,doc-per-day=3,force_clean=True,reload_data=False,nodes_init=1,services_init=kv;n1ql;index Test Input params: {'doc-per-day': '3', 'conf_file': './4node.conf', 'num_nodes': 4, 'cluster_name': '4node', 'reload_data': 'False', 'force_clean': 'True', 'ini': './4node.ini', 'case_number': 25, 'services_init': 'kv;n1ql;index', 'logs_folder': '/testrunner/logs/testrunner-18-Jan-25_21-16-28/test_25', 'spec': '4node', 'nodes_init': '1'} [2018-01-25 21:54:13,642] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 21:54:13,776] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 21:54:14,189] - [rest_client:1964] INFO - Node version in cluster 5.0.1-5003-enterprise [2018-01-25 21:54:14,202] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:54:14,202] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:54:14,203] - [basetestcase:201] INFO - ============== basetestcase setup was started for test #25 test_primary_create_delete_index============== [2018-01-25 21:54:14,203] - [basetestcase:230] INFO - any cluster operation in setup will be skipped [2018-01-25 21:54:14,203] - [basetestcase:233] INFO - ============== basetestcase setup was finished for test #25 test_primary_create_delete_index ============== [2018-01-25 21:54:14,204] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 21:54:14,334] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 21:54:14,658] - [rest_client:1964] INFO - Node version in cluster 5.0.1-5003-enterprise [2018-01-25 21:54:14,665] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:54:14,665] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:54:14,665] - [basetestcase:201] INFO - ============== basetestcase setup was started for test #25 test_primary_create_delete_index============== [2018-01-25 21:54:14,679] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.8 [2018-01-25 21:54:14,679] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:54:16,689] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.6 [2018-01-25 21:54:16,689] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:54:18,715] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.7 [2018-01-25 21:54:18,716] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:54:20,736] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.5 [2018-01-25 21:54:20,738] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:54:22,754] - [cluster_helper:254] INFO - rebalancing all nodes in order to remove nodes [2018-01-25 21:54:22,756] - [rest_client:1423] INFO - rebalance params : {'password': 'password', 'ejectedNodes': u'ns_1@172.17.0.7,ns_1@172.17.0.5,ns_1@172.17.0.6', 'user': 'Administrator', 'knownNodes': u'ns_1@cb-example-0003.cb-example.default.svc,ns_1@172.17.0.7,ns_1@172.17.0.5,ns_1@172.17.0.6'} [2018-01-25 21:54:22,770] - [rest_client:807] ERROR - POST http://172.17.0.8:8091/controller/rebalance body: password=password&ejectedNodes=ns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6&user=Administrator&knownNodes=ns_1%40cb-example-0003.cb-example.default.svc%2Cns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6 headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 400 reason: status: 400, content: No active nodes left No active nodes left auth: Administrator:password [2018-01-25 21:54:22,772] - [rest_client:1430] ERROR - rebalance operation failed: No active nodes left test_primary_create_delete_index (tuqquery.tuq_index.QueriesViewsTests) ... Traceback (most recent call last): File "pytests/basetestcase.py", line 203, in setUp self._cluster_cleanup() File "pytests/basetestcase.py", line 494, in _cluster_cleanup ClusterOperationHelper.cleanup_cluster(self.servers, master=self.master) File "lib/membase/helper/cluster_helper.py", line 259, in cleanup_cluster wait_for_rebalance=wait_for_rebalance) File "lib/membase/api/rest_client.py", line 110, in remove_nodes self.rest.rebalance(knownNodes, ejectedNodes) File "lib/membase/api/rest_client.py", line 1433, in rebalance parameters=params) InvalidArgumentException: controller/rebalance with error message No active nodes left failed when invoked with parameters: password=password&ejectedNodes=ns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6&user=Administrator&knownNodes=ns_1%40cb-example-0003.cb-example.default.svc%2Cns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6 ERROR ====================================================================== ERROR: test_primary_create_delete_index (tuqquery.tuq_index.QueriesViewsTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "pytests/tuqquery/tuq_index.py", line 13, in setUp super(QueriesViewsTests, self).setUp() File "pytests/tuqquery/tuq_sanity.py", line 17, in setUp super(QuerySanityTests, self).setUp() File "pytests/tuqquery/tuq.py", line 40, in setUp super(QueryTests, self).setUp() File "pytests/basetestcase.py", line 352, in setUp self.fail(e) AssertionError: controller/rebalance with error message No active nodes left failed when invoked with parameters: password=password&ejectedNodes=ns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6&user=Administrator&knownNodes=ns_1%40cb-example-0003.cb-example.default.svc%2Cns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6 ---------------------------------------------------------------------- Ran 1 test in 9.133s FAILED (errors=1) Cluster instance shutdown with force summary so far suite recreatebuckettests.RecreateMembaseBuckets , pass 0 , fail 1 failures so far... recreatebuckettests.RecreateMembaseBuckets.test_default_moxi summary so far suite deletebuckettests.DeleteMembaseBuckets , pass 1 , fail 0 summary so far suite createbuckettests.CreateMembaseBucketsTests , pass 1 , fail 4 failures so far... createbuckettests.CreateMembaseBucketsTests.test_default_moxi createbuckettests.CreateMembaseBucketsTests.test_default_on_non_default_port createbuckettests.CreateMembaseBucketsTests.test_non_default_case_sensitive_different_port createbuckettests.CreateMembaseBucketsTests.test_two_replica summary so far suite setgettests.MembaseBucket , pass 1 , fail 0 summary so far suite expirytests.ExpiryTests , pass 1 , fail 0 summary so far suite memcapable.GetlTests , pass 2 , fail 0 summary so far suite memorysanitytests.MemorySanity , pass 1 , fail 0 summary so far suite drainratetests.DrainRateTests , pass 1 , fail 0 summary so far suite view.viewquerytests.ViewQueryTests , pass 1 , fail 0 summary so far suite view.createdeleteview.CreateDeleteViewTests , pass 1 , fail 0 summary so far suite rebalance.rebalancein.RebalanceInTests , pass 0 , fail 1 failures so far... rebalance.rebalancein.RebalanceInTests.rebalance_in_with_ops summary so far suite rebalance.rebalanceout.RebalanceOutTests , pass 0 , fail 1 failures so far... rebalance.rebalanceout.RebalanceOutTests.rebalance_out_with_ops summary so far suite failover.failovertests.FailoverTests , pass 0 , fail 1 failures so far... failover.failovertests.FailoverTests.test_failover_normal summary so far suite CCCP.CCCP , pass 0 , fail 2 failures so far... CCCP.CCCP.test_get_config_client CCCP.CCCP.test_not_my_vbucket_config summary so far suite flush.bucketflush.BucketFlushTests , pass 0 , fail 1 failures so far... flush.bucketflush.BucketFlushTests.bucketflush_with_data_ops_moxi summary so far suite security.audittest.auditTest , pass 0 , fail 1 failures so far... security.audittest.auditTest.test_bucketEvents summary so far suite tuqquery.tuq_precedence.PrecedenceTests , pass 0 , fail 2 failures so far... tuqquery.tuq_precedence.PrecedenceTests.test_case_and_like tuqquery.tuq_precedence.PrecedenceTests.test_case_and_like summary so far suite tuqquery.tuq_index.QueriesViewsTests , pass 0 , fail 1 failures so far... tuqquery.tuq_index.QueriesViewsTests.test_primary_create_delete_index testrunner logs, diags and results are available under /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_25 Logs will be stored at /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_26 ./testrunner -i ./4node.ini -t tuqquery.tuq_index.QueriesViewsTests.test_primary_create_delete_index,doc-per-day=2,primary_indx_type=GSI,reload_data=False,force_clean=True,nodes_init=1 Test Input params: {'doc-per-day': '2', 'primary_indx_type': 'GSI', 'conf_file': './4node.conf', 'num_nodes': 4, 'cluster_name': '4node', 'reload_data': 'False', 'force_clean': 'True', 'ini': './4node.ini', 'case_number': 26, 'logs_folder': '/testrunner/logs/testrunner-18-Jan-25_21-16-28/test_26', 'spec': '4node', 'nodes_init': '1'} [2018-01-25 21:54:22,787] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 21:54:22,918] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 21:54:23,327] - [rest_client:1964] INFO - Node version in cluster 5.0.1-5003-enterprise [2018-01-25 21:54:23,338] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:54:23,338] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:54:23,338] - [basetestcase:201] INFO - ============== basetestcase setup was started for test #26 test_primary_create_delete_index============== [2018-01-25 21:54:23,339] - [basetestcase:230] INFO - any cluster operation in setup will be skipped [2018-01-25 21:54:23,340] - [basetestcase:233] INFO - ============== basetestcase setup was finished for test #26 test_primary_create_delete_index ============== [2018-01-25 21:54:23,341] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 21:54:23,431] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 21:54:23,821] - [rest_client:1964] INFO - Node version in cluster 5.0.1-5003-enterprise [2018-01-25 21:54:23,830] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:54:23,830] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:54:23,830] - [basetestcase:201] INFO - ============== basetestcase setup was started for test #26 test_primary_create_delete_index============== [2018-01-25 21:54:23,847] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.8 [2018-01-25 21:54:23,847] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:54:25,861] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.6 [2018-01-25 21:54:25,862] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:54:27,883] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.7 [2018-01-25 21:54:27,884] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:54:29,895] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.5 [2018-01-25 21:54:29,896] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:54:31,919] - [cluster_helper:254] INFO - rebalancing all nodes in order to remove nodes [2018-01-25 21:54:31,924] - [rest_client:1423] INFO - rebalance params : {'password': 'password', 'ejectedNodes': u'ns_1@172.17.0.7,ns_1@172.17.0.5,ns_1@172.17.0.6', 'user': 'Administrator', 'knownNodes': u'ns_1@cb-example-0003.cb-example.default.svc,ns_1@172.17.0.7,ns_1@172.17.0.5,ns_1@172.17.0.6'} [2018-01-25 21:54:31,929] - [rest_client:807] ERROR - POST http://172.17.0.8:8091/controller/rebalance body: password=password&ejectedNodes=ns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6&user=Administrator&knownNodes=ns_1%40cb-example-0003.cb-example.default.svc%2Cns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6 headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 400 reason: status: 400, content: No active nodes left No active nodes left auth: Administrator:password [2018-01-25 21:54:31,929] - [rest_client:1430] ERROR - rebalance operation failed: No active nodes left test_primary_create_delete_index (tuqquery.tuq_index.QueriesViewsTests) ... Traceback (most recent call last): File "pytests/basetestcase.py", line 203, in setUp self._cluster_cleanup() File "pytests/basetestcase.py", line 494, in _cluster_cleanup ClusterOperationHelper.cleanup_cluster(self.servers, master=self.master) File "lib/membase/helper/cluster_helper.py", line 259, in cleanup_cluster wait_for_rebalance=wait_for_rebalance) File "lib/membase/api/rest_client.py", line 110, in remove_nodes self.rest.rebalance(knownNodes, ejectedNodes) File "lib/membase/api/rest_client.py", line 1433, in rebalance parameters=params) InvalidArgumentException: controller/rebalance with error message No active nodes left failed when invoked with parameters: password=password&ejectedNodes=ns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6&user=Administrator&knownNodes=ns_1%40cb-example-0003.cb-example.default.svc%2Cns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6 ERROR ====================================================================== ERROR: test_primary_create_delete_index (tuqquery.tuq_index.QueriesViewsTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "pytests/tuqquery/tuq_index.py", line 13, in setUp super(QueriesViewsTests, self).setUp() File "pytests/tuqquery/tuq_sanity.py", line 17, in setUp super(QuerySanityTests, self).setUp() File "pytests/tuqquery/tuq.py", line 40, in setUp super(QueryTests, self).setUp() File "pytests/basetestcase.py", line 352, in setUp self.fail(e) AssertionError: controller/rebalance with error message No active nodes left failed when invoked with parameters: password=password&ejectedNodes=ns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6&user=Administrator&knownNodes=ns_1%40cb-example-0003.cb-example.default.svc%2Cns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6 ---------------------------------------------------------------------- Ran 1 test in 9.144s FAILED (errors=1) Cluster instance shutdown with force summary so far suite recreatebuckettests.RecreateMembaseBuckets , pass 0 , fail 1 failures so far... recreatebuckettests.RecreateMembaseBuckets.test_default_moxi summary so far suite deletebuckettests.DeleteMembaseBuckets , pass 1 , fail 0 summary so far suite createbuckettests.CreateMembaseBucketsTests , pass 1 , fail 4 failures so far... createbuckettests.CreateMembaseBucketsTests.test_default_moxi createbuckettests.CreateMembaseBucketsTests.test_default_on_non_default_port createbuckettests.CreateMembaseBucketsTests.test_non_default_case_sensitive_different_port createbuckettests.CreateMembaseBucketsTests.test_two_replica summary so far suite setgettests.MembaseBucket , pass 1 , fail 0 summary so far suite expirytests.ExpiryTests , pass 1 , fail 0 summary so far suite memcapable.GetlTests , pass 2 , fail 0 summary so far suite memorysanitytests.MemorySanity , pass 1 , fail 0 summary so far suite drainratetests.DrainRateTests , pass 1 , fail 0 summary so far suite view.viewquerytests.ViewQueryTests , pass 1 , fail 0 summary so far suite view.createdeleteview.CreateDeleteViewTests , pass 1 , fail 0 summary so far suite rebalance.rebalancein.RebalanceInTests , pass 0 , fail 1 failures so far... rebalance.rebalancein.RebalanceInTests.rebalance_in_with_ops summary so far suite rebalance.rebalanceout.RebalanceOutTests , pass 0 , fail 1 failures so far... rebalance.rebalanceout.RebalanceOutTests.rebalance_out_with_ops summary so far suite failover.failovertests.FailoverTests , pass 0 , fail 1 failures so far... failover.failovertests.FailoverTests.test_failover_normal summary so far suite CCCP.CCCP , pass 0 , fail 2 failures so far... CCCP.CCCP.test_get_config_client CCCP.CCCP.test_not_my_vbucket_config summary so far suite flush.bucketflush.BucketFlushTests , pass 0 , fail 1 failures so far... flush.bucketflush.BucketFlushTests.bucketflush_with_data_ops_moxi summary so far suite security.audittest.auditTest , pass 0 , fail 1 failures so far... security.audittest.auditTest.test_bucketEvents summary so far suite tuqquery.tuq_precedence.PrecedenceTests , pass 0 , fail 2 failures so far... tuqquery.tuq_precedence.PrecedenceTests.test_case_and_like tuqquery.tuq_precedence.PrecedenceTests.test_case_and_like summary so far suite tuqquery.tuq_index.QueriesViewsTests , pass 0 , fail 2 failures so far... tuqquery.tuq_index.QueriesViewsTests.test_primary_create_delete_index tuqquery.tuq_index.QueriesViewsTests.test_primary_create_delete_index testrunner logs, diags and results are available under /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_26 Logs will be stored at /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_27 ./testrunner -i ./4node.ini -t tuqquery.tuq_index.QueriesViewsTests.test_explain_index_attr,force_clean=True,reload_data=False,doc-per-day=2,nodes_init=1 Test Input params: {'doc-per-day': '2', 'conf_file': './4node.conf', 'num_nodes': 4, 'cluster_name': '4node', 'reload_data': 'False', 'force_clean': 'True', 'ini': './4node.ini', 'case_number': 27, 'logs_folder': '/testrunner/logs/testrunner-18-Jan-25_21-16-28/test_27', 'spec': '4node', 'nodes_init': '1'} [2018-01-25 21:54:31,943] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 21:54:32,074] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 21:54:32,538] - [rest_client:1964] INFO - Node version in cluster 5.0.1-5003-enterprise [2018-01-25 21:54:32,546] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:54:32,546] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:54:32,546] - [basetestcase:201] INFO - ============== basetestcase setup was started for test #27 test_explain_index_attr============== [2018-01-25 21:54:32,547] - [basetestcase:230] INFO - any cluster operation in setup will be skipped [2018-01-25 21:54:32,547] - [basetestcase:233] INFO - ============== basetestcase setup was finished for test #27 test_explain_index_attr ============== [2018-01-25 21:54:32,548] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 21:54:32,679] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 21:54:33,134] - [rest_client:1964] INFO - Node version in cluster 5.0.1-5003-enterprise [2018-01-25 21:54:33,145] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:54:33,145] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:54:33,145] - [basetestcase:201] INFO - ============== basetestcase setup was started for test #27 test_explain_index_attr============== [2018-01-25 21:54:33,169] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.8 [2018-01-25 21:54:33,169] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:54:35,185] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.6 [2018-01-25 21:54:35,186] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:54:37,201] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.7 [2018-01-25 21:54:37,202] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:54:39,213] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.5 [2018-01-25 21:54:39,214] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:54:41,240] - [cluster_helper:254] INFO - rebalancing all nodes in order to remove nodes [2018-01-25 21:54:41,242] - [rest_client:1423] INFO - rebalance params : {'password': 'password', 'ejectedNodes': u'ns_1@172.17.0.7,ns_1@172.17.0.5,ns_1@172.17.0.6', 'user': 'Administrator', 'knownNodes': u'ns_1@cb-example-0003.cb-example.default.svc,ns_1@172.17.0.7,ns_1@172.17.0.5,ns_1@172.17.0.6'} [2018-01-25 21:54:41,248] - [rest_client:807] ERROR - POST http://172.17.0.8:8091/controller/rebalance body: password=password&ejectedNodes=ns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6&user=Administrator&knownNodes=ns_1%40cb-example-0003.cb-example.default.svc%2Cns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6 headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 400 reason: status: 400, content: No active nodes left No active nodes left auth: Administrator:password [2018-01-25 21:54:41,249] - [rest_client:1430] ERROR - rebalance operation failed: No active nodes left test_explain_index_attr (tuqquery.tuq_index.QueriesViewsTests) ... Traceback (most recent call last): File "pytests/basetestcase.py", line 203, in setUp self._cluster_cleanup() File "pytests/basetestcase.py", line 494, in _cluster_cleanup ClusterOperationHelper.cleanup_cluster(self.servers, master=self.master) File "lib/membase/helper/cluster_helper.py", line 259, in cleanup_cluster wait_for_rebalance=wait_for_rebalance) File "lib/membase/api/rest_client.py", line 110, in remove_nodes self.rest.rebalance(knownNodes, ejectedNodes) File "lib/membase/api/rest_client.py", line 1433, in rebalance parameters=params) InvalidArgumentException: controller/rebalance with error message No active nodes left failed when invoked with parameters: password=password&ejectedNodes=ns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6&user=Administrator&knownNodes=ns_1%40cb-example-0003.cb-example.default.svc%2Cns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6 ERROR ====================================================================== ERROR: test_explain_index_attr (tuqquery.tuq_index.QueriesViewsTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "pytests/tuqquery/tuq_index.py", line 13, in setUp super(QueriesViewsTests, self).setUp() File "pytests/tuqquery/tuq_sanity.py", line 17, in setUp super(QuerySanityTests, self).setUp() File "pytests/tuqquery/tuq.py", line 40, in setUp super(QueryTests, self).setUp() File "pytests/basetestcase.py", line 352, in setUp self.fail(e) AssertionError: controller/rebalance with error message No active nodes left failed when invoked with parameters: password=password&ejectedNodes=ns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6&user=Administrator&knownNodes=ns_1%40cb-example-0003.cb-example.default.svc%2Cns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6 ---------------------------------------------------------------------- Ran 1 test in 9.308s FAILED (errors=1) Cluster instance shutdown with force summary so far suite recreatebuckettests.RecreateMembaseBuckets , pass 0 , fail 1 failures so far... recreatebuckettests.RecreateMembaseBuckets.test_default_moxi summary so far suite deletebuckettests.DeleteMembaseBuckets , pass 1 , fail 0 summary so far suite createbuckettests.CreateMembaseBucketsTests , pass 1 , fail 4 failures so far... createbuckettests.CreateMembaseBucketsTests.test_default_moxi createbuckettests.CreateMembaseBucketsTests.test_default_on_non_default_port createbuckettests.CreateMembaseBucketsTests.test_non_default_case_sensitive_different_port createbuckettests.CreateMembaseBucketsTests.test_two_replica summary so far suite setgettests.MembaseBucket , pass 1 , fail 0 summary so far suite expirytests.ExpiryTests , pass 1 , fail 0 summary so far suite memcapable.GetlTests , pass 2 , fail 0 summary so far suite memorysanitytests.MemorySanity , pass 1 , fail 0 summary so far suite drainratetests.DrainRateTests , pass 1 , fail 0 summary so far suite view.viewquerytests.ViewQueryTests , pass 1 , fail 0 summary so far suite view.createdeleteview.CreateDeleteViewTests , pass 1 , fail 0 summary so far suite rebalance.rebalancein.RebalanceInTests , pass 0 , fail 1 failures so far... rebalance.rebalancein.RebalanceInTests.rebalance_in_with_ops summary so far suite rebalance.rebalanceout.RebalanceOutTests , pass 0 , fail 1 failures so far... rebalance.rebalanceout.RebalanceOutTests.rebalance_out_with_ops summary so far suite failover.failovertests.FailoverTests , pass 0 , fail 1 failures so far... failover.failovertests.FailoverTests.test_failover_normal summary so far suite CCCP.CCCP , pass 0 , fail 2 failures so far... CCCP.CCCP.test_get_config_client CCCP.CCCP.test_not_my_vbucket_config summary so far suite flush.bucketflush.BucketFlushTests , pass 0 , fail 1 failures so far... flush.bucketflush.BucketFlushTests.bucketflush_with_data_ops_moxi summary so far suite security.audittest.auditTest , pass 0 , fail 1 failures so far... security.audittest.auditTest.test_bucketEvents summary so far suite tuqquery.tuq_precedence.PrecedenceTests , pass 0 , fail 2 failures so far... tuqquery.tuq_precedence.PrecedenceTests.test_case_and_like tuqquery.tuq_precedence.PrecedenceTests.test_case_and_like summary so far suite tuqquery.tuq_index.QueriesViewsTests , pass 0 , fail 3 failures so far... tuqquery.tuq_index.QueriesViewsTests.test_primary_create_delete_index tuqquery.tuq_index.QueriesViewsTests.test_primary_create_delete_index tuqquery.tuq_index.QueriesViewsTests.test_explain_index_attr testrunner logs, diags and results are available under /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_27 Logs will be stored at /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_28 ./testrunner -i ./4node.ini -t tuqquery.tuq_dml.DMLQueryTests.test_sanity,force_clean=True,reload_data=False,nodes_init=1,skip_load=True Test Input params: {'conf_file': './4node.conf', 'num_nodes': 4, 'cluster_name': '4node', 'reload_data': 'False', 'force_clean': 'True', 'skip_load': 'True', 'case_number': 28, 'logs_folder': '/testrunner/logs/testrunner-18-Jan-25_21-16-28/test_28', 'spec': '4node', 'nodes_init': '1', 'ini': './4node.ini'} [2018-01-25 21:54:41,314] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 21:54:41,419] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 21:54:41,904] - [rest_client:1964] INFO - Node version in cluster 5.0.1-5003-enterprise [2018-01-25 21:54:41,919] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:54:41,919] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:54:41,920] - [basetestcase:201] INFO - ============== basetestcase setup was started for test #28 test_sanity============== [2018-01-25 21:54:41,920] - [basetestcase:230] INFO - any cluster operation in setup will be skipped [2018-01-25 21:54:41,921] - [basetestcase:233] INFO - ============== basetestcase setup was finished for test #28 test_sanity ============== [2018-01-25 21:54:41,924] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 21:54:42,059] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 21:54:42,730] - [rest_client:1964] INFO - Node version in cluster 5.0.1-5003-enterprise [2018-01-25 21:54:42,741] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:54:42,741] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:54:42,741] - [basetestcase:201] INFO - ============== basetestcase setup was started for test #28 test_sanity============== [2018-01-25 21:54:42,768] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.8 [2018-01-25 21:54:42,769] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:54:44,783] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.6 [2018-01-25 21:54:44,783] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:54:46,798] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.7 [2018-01-25 21:54:46,798] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:54:48,814] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.5 [2018-01-25 21:54:48,814] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:54:50,837] - [cluster_helper:254] INFO - rebalancing all nodes in order to remove nodes [2018-01-25 21:54:50,840] - [rest_client:1423] INFO - rebalance params : {'password': 'password', 'ejectedNodes': u'ns_1@172.17.0.7,ns_1@172.17.0.5,ns_1@172.17.0.6', 'user': 'Administrator', 'knownNodes': u'ns_1@cb-example-0003.cb-example.default.svc,ns_1@172.17.0.7,ns_1@172.17.0.5,ns_1@172.17.0.6'} [2018-01-25 21:54:50,846] - [rest_client:807] ERROR - POST http://172.17.0.8:8091/controller/rebalance body: password=password&ejectedNodes=ns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6&user=Administrator&knownNodes=ns_1%40cb-example-0003.cb-example.default.svc%2Cns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6 headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 400 reason: status: 400, content: No active nodes left No active nodes left auth: Administrator:password [2018-01-25 21:54:50,847] - [rest_client:1430] ERROR - rebalance operation failed: No active nodes left test_sanity (tuqquery.tuq_dml.DMLQueryTests) ... Traceback (most recent call last): File "pytests/basetestcase.py", line 203, in setUp self._cluster_cleanup() File "pytests/basetestcase.py", line 494, in _cluster_cleanup ClusterOperationHelper.cleanup_cluster(self.servers, master=self.master) File "lib/membase/helper/cluster_helper.py", line 259, in cleanup_cluster wait_for_rebalance=wait_for_rebalance) File "lib/membase/api/rest_client.py", line 110, in remove_nodes self.rest.rebalance(knownNodes, ejectedNodes) File "lib/membase/api/rest_client.py", line 1433, in rebalance parameters=params) InvalidArgumentException: controller/rebalance with error message No active nodes left failed when invoked with parameters: password=password&ejectedNodes=ns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6&user=Administrator&knownNodes=ns_1%40cb-example-0003.cb-example.default.svc%2Cns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6 ERROR ====================================================================== ERROR: test_sanity (tuqquery.tuq_dml.DMLQueryTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "pytests/tuqquery/tuq_dml.py", line 9, in setUp super(DMLQueryTests, self).setUp() File "pytests/tuqquery/tuq.py", line 40, in setUp super(QueryTests, self).setUp() File "pytests/basetestcase.py", line 352, in setUp self.fail(e) AssertionError: controller/rebalance with error message No active nodes left failed when invoked with parameters: password=password&ejectedNodes=ns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6&user=Administrator&knownNodes=ns_1%40cb-example-0003.cb-example.default.svc%2Cns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6 ---------------------------------------------------------------------- Ran 1 test in 9.545s FAILED (errors=1) Cluster instance shutdown with force summary so far suite recreatebuckettests.RecreateMembaseBuckets , pass 0 , fail 1 failures so far... recreatebuckettests.RecreateMembaseBuckets.test_default_moxi summary so far suite deletebuckettests.DeleteMembaseBuckets , pass 1 , fail 0 summary so far suite createbuckettests.CreateMembaseBucketsTests , pass 1 , fail 4 failures so far... createbuckettests.CreateMembaseBucketsTests.test_default_moxi createbuckettests.CreateMembaseBucketsTests.test_default_on_non_default_port createbuckettests.CreateMembaseBucketsTests.test_non_default_case_sensitive_different_port createbuckettests.CreateMembaseBucketsTests.test_two_replica summary so far suite setgettests.MembaseBucket , pass 1 , fail 0 summary so far suite expirytests.ExpiryTests , pass 1 , fail 0 summary so far suite memcapable.GetlTests , pass 2 , fail 0 summary so far suite memorysanitytests.MemorySanity , pass 1 , fail 0 summary so far suite drainratetests.DrainRateTests , pass 1 , fail 0 summary so far suite view.viewquerytests.ViewQueryTests , pass 1 , fail 0 summary so far suite view.createdeleteview.CreateDeleteViewTests , pass 1 , fail 0 summary so far suite rebalance.rebalancein.RebalanceInTests , pass 0 , fail 1 failures so far... rebalance.rebalancein.RebalanceInTests.rebalance_in_with_ops summary so far suite rebalance.rebalanceout.RebalanceOutTests , pass 0 , fail 1 failures so far... rebalance.rebalanceout.RebalanceOutTests.rebalance_out_with_ops summary so far suite failover.failovertests.FailoverTests , pass 0 , fail 1 failures so far... failover.failovertests.FailoverTests.test_failover_normal summary so far suite CCCP.CCCP , pass 0 , fail 2 failures so far... CCCP.CCCP.test_get_config_client CCCP.CCCP.test_not_my_vbucket_config summary so far suite flush.bucketflush.BucketFlushTests , pass 0 , fail 1 failures so far... flush.bucketflush.BucketFlushTests.bucketflush_with_data_ops_moxi summary so far suite security.audittest.auditTest , pass 0 , fail 1 failures so far... security.audittest.auditTest.test_bucketEvents summary so far suite tuqquery.tuq_precedence.PrecedenceTests , pass 0 , fail 2 failures so far... tuqquery.tuq_precedence.PrecedenceTests.test_case_and_like tuqquery.tuq_precedence.PrecedenceTests.test_case_and_like summary so far suite tuqquery.tuq_index.QueriesViewsTests , pass 0 , fail 3 failures so far... tuqquery.tuq_index.QueriesViewsTests.test_primary_create_delete_index tuqquery.tuq_index.QueriesViewsTests.test_primary_create_delete_index tuqquery.tuq_index.QueriesViewsTests.test_explain_index_attr summary so far suite tuqquery.tuq_dml.DMLQueryTests , pass 0 , fail 1 failures so far... tuqquery.tuq_dml.DMLQueryTests.test_sanity testrunner logs, diags and results are available under /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_28 Logs will be stored at /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_29 ./testrunner -i ./4node.ini -t 2i.indexscans_2i.SecondaryIndexingScanTests.test_multi_create_query_explain_drop_index,groups=simple,doc-per-day=10,dataset=default,use_gsi_for_primary=true,reset_services=True Test Input params: {'doc-per-day': '10', 'use_gsi_for_primary': 'true', 'case_number': 29, 'reset_services': 'True', 'dataset': 'default', 'cluster_name': '4node', 'ini': './4node.ini', 'groups': 'simple', 'conf_file': './4node.conf', 'num_nodes': 4, 'logs_folder': '/testrunner/logs/testrunner-18-Jan-25_21-16-28/test_29', 'spec': '4node'} [2018-01-25 21:54:50,921] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 21:54:51,117] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 21:54:51,710] - [rest_client:1964] INFO - Node version in cluster 5.0.1-5003-enterprise [2018-01-25 21:54:51,723] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:54:51,723] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:54:51,724] - [basetestcase:201] INFO - ============== basetestcase setup was started for test #29 test_multi_create_query_explain_drop_index============== [2018-01-25 21:54:51,749] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.8 [2018-01-25 21:54:51,750] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:54:53,758] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.6 [2018-01-25 21:54:53,759] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:54:55,767] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.7 [2018-01-25 21:54:55,767] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:54:57,777] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.5 [2018-01-25 21:54:57,778] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:54:59,796] - [cluster_helper:254] INFO - rebalancing all nodes in order to remove nodes [2018-01-25 21:54:59,798] - [rest_client:1423] INFO - rebalance params : {'password': 'password', 'ejectedNodes': u'ns_1@172.17.0.7,ns_1@172.17.0.5,ns_1@172.17.0.6', 'user': 'Administrator', 'knownNodes': u'ns_1@cb-example-0003.cb-example.default.svc,ns_1@172.17.0.7,ns_1@172.17.0.5,ns_1@172.17.0.6'} [2018-01-25 21:54:59,801] - [rest_client:807] ERROR - POST http://172.17.0.8:8091/controller/rebalance body: password=password&ejectedNodes=ns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6&user=Administrator&knownNodes=ns_1%40cb-example-0003.cb-example.default.svc%2Cns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6 headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 400 reason: status: 400, content: No active nodes left No active nodes left auth: Administrator:password [2018-01-25 21:54:59,803] - [rest_client:1430] ERROR - rebalance operation failed: No active nodes left test_multi_create_query_explain_drop_index (2i.indexscans_2i.SecondaryIndexingScanTests) ... Traceback (most recent call last): File "pytests/basetestcase.py", line 203, in setUp self._cluster_cleanup() File "pytests/basetestcase.py", line 494, in _cluster_cleanup ClusterOperationHelper.cleanup_cluster(self.servers, master=self.master) File "lib/membase/helper/cluster_helper.py", line 259, in cleanup_cluster wait_for_rebalance=wait_for_rebalance) File "lib/membase/api/rest_client.py", line 110, in remove_nodes self.rest.rebalance(knownNodes, ejectedNodes) File "lib/membase/api/rest_client.py", line 1433, in rebalance parameters=params) InvalidArgumentException: controller/rebalance with error message No active nodes left failed when invoked with parameters: password=password&ejectedNodes=ns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6&user=Administrator&knownNodes=ns_1%40cb-example-0003.cb-example.default.svc%2Cns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6 ERROR ====================================================================== ERROR: test_multi_create_query_explain_drop_index (2i.indexscans_2i.SecondaryIndexingScanTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "pytests/2i/indexscans_2i.py", line 11, in setUp super(SecondaryIndexingScanTests, self).setUp() File "pytests/2i/base_2i.py", line 15, in setUp super(BaseSecondaryIndexingTests, self).setUp() File "pytests/2i/newtuq.py", line 10, in setUp super(QueryTests, self).setUp() File "pytests/basetestcase.py", line 352, in setUp self.fail(e) AssertionError: controller/rebalance with error message No active nodes left failed when invoked with parameters: password=password&ejectedNodes=ns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6&user=Administrator&knownNodes=ns_1%40cb-example-0003.cb-example.default.svc%2Cns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6 ---------------------------------------------------------------------- Ran 1 test in 8.902s FAILED (errors=1) Cluster instance shutdown with force summary so far suite recreatebuckettests.RecreateMembaseBuckets , pass 0 , fail 1 failures so far... recreatebuckettests.RecreateMembaseBuckets.test_default_moxi summary so far suite deletebuckettests.DeleteMembaseBuckets , pass 1 , fail 0 summary so far suite createbuckettests.CreateMembaseBucketsTests , pass 1 , fail 4 failures so far... createbuckettests.CreateMembaseBucketsTests.test_default_moxi createbuckettests.CreateMembaseBucketsTests.test_default_on_non_default_port createbuckettests.CreateMembaseBucketsTests.test_non_default_case_sensitive_different_port createbuckettests.CreateMembaseBucketsTests.test_two_replica summary so far suite setgettests.MembaseBucket , pass 1 , fail 0 summary so far suite expirytests.ExpiryTests , pass 1 , fail 0 summary so far suite memcapable.GetlTests , pass 2 , fail 0 summary so far suite memorysanitytests.MemorySanity , pass 1 , fail 0 summary so far suite drainratetests.DrainRateTests , pass 1 , fail 0 summary so far suite view.viewquerytests.ViewQueryTests , pass 1 , fail 0 summary so far suite view.createdeleteview.CreateDeleteViewTests , pass 1 , fail 0 summary so far suite rebalance.rebalancein.RebalanceInTests , pass 0 , fail 1 failures so far... rebalance.rebalancein.RebalanceInTests.rebalance_in_with_ops summary so far suite rebalance.rebalanceout.RebalanceOutTests , pass 0 , fail 1 failures so far... rebalance.rebalanceout.RebalanceOutTests.rebalance_out_with_ops summary so far suite failover.failovertests.FailoverTests , pass 0 , fail 1 failures so far... failover.failovertests.FailoverTests.test_failover_normal summary so far suite CCCP.CCCP , pass 0 , fail 2 failures so far... CCCP.CCCP.test_get_config_client CCCP.CCCP.test_not_my_vbucket_config summary so far suite flush.bucketflush.BucketFlushTests , pass 0 , fail 1 failures so far... flush.bucketflush.BucketFlushTests.bucketflush_with_data_ops_moxi summary so far suite security.audittest.auditTest , pass 0 , fail 1 failures so far... security.audittest.auditTest.test_bucketEvents summary so far suite tuqquery.tuq_precedence.PrecedenceTests , pass 0 , fail 2 failures so far... tuqquery.tuq_precedence.PrecedenceTests.test_case_and_like tuqquery.tuq_precedence.PrecedenceTests.test_case_and_like summary so far suite tuqquery.tuq_index.QueriesViewsTests , pass 0 , fail 3 failures so far... tuqquery.tuq_index.QueriesViewsTests.test_primary_create_delete_index tuqquery.tuq_index.QueriesViewsTests.test_primary_create_delete_index tuqquery.tuq_index.QueriesViewsTests.test_explain_index_attr summary so far suite tuqquery.tuq_dml.DMLQueryTests , pass 0 , fail 1 failures so far... tuqquery.tuq_dml.DMLQueryTests.test_sanity summary so far suite 2i.indexscans_2i.SecondaryIndexingScanTests , pass 0 , fail 1 failures so far... 2i.indexscans_2i.SecondaryIndexingScanTests.test_multi_create_query_explain_drop_index testrunner logs, diags and results are available under /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_29 Logs will be stored at /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_30 ./testrunner -i ./4node.ini -t xdcr.uniXDCR.unidirectional.load_with_ops,items=5000,expires=20,ctopology=chain,rdirection=unidirection,update=C1,delete=C1 Test Input params: {'items': '5000', 'expires': '20', 'update': 'C1', 'conf_file': './4node.conf', 'num_nodes': 4, 'cluster_name': '4node', 'ctopology': 'chain', 'rdirection': 'unidirection', 'ini': './4node.ini', 'case_number': 30, 'logs_folder': '/testrunner/logs/testrunner-18-Jan-25_21-16-28/test_30', 'spec': '4node', 'delete': 'C1'} 2018-01-25 21:54:59,907 - root - WARNING - Can not import concurrent module. Data for each server will be loaded/retrieved sequentially 2018-01-25 21:54:59,917 - root - INFO - /diag/eval status on 172.17.0.8:8091: True content: "/opt/couchbase/var/lib/couchbase/logs" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))). 2018-01-25 21:54:59,918 - root - INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: 2018-01-25 21:55:00,050 - root - INFO - Connected to 172.17.0.8 2018-01-25 21:55:01,180 - root - INFO - running command.raw on 172.17.0.8: zgrep "panic:" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l 2018-01-25 21:55:01,231 - root - INFO - command executed successfully 2018-01-25 21:55:01,232 - root - INFO - 0 2018-01-25 21:55:01,349 - root - INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: 2018-01-25 21:55:01,432 - root - INFO - Connected to 172.17.0.8 2018-01-25 21:55:01,848 - root - INFO - running command.raw on 172.17.0.8: zgrep "non-recoverable error from xmem client. response status=KEY_ENOENT" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l 2018-01-25 21:55:01,924 - root - INFO - command executed successfully 2018-01-25 21:55:01,925 - root - INFO - 0 2018-01-25 21:55:02,046 - root - INFO - connecting to 172.17.0.6 with username:root password:couchbase ssh_key: 2018-01-25 21:55:02,234 - root - INFO - Connected to 172.17.0.6 2018-01-25 21:55:02,687 - root - INFO - running command.raw on 172.17.0.6: zgrep "panic:" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l 2018-01-25 21:55:02,762 - root - INFO - command executed successfully 2018-01-25 21:55:02,762 - root - INFO - 0 2018-01-25 21:55:02,879 - root - INFO - connecting to 172.17.0.6 with username:root password:couchbase ssh_key: 2018-01-25 21:55:03,015 - root - INFO - Connected to 172.17.0.6 2018-01-25 21:55:03,463 - root - INFO - running command.raw on 172.17.0.6: zgrep "non-recoverable error from xmem client. response status=KEY_ENOENT" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l 2018-01-25 21:55:03,540 - root - INFO - command executed successfully 2018-01-25 21:55:03,540 - root - INFO - 0 2018-01-25 21:55:03,656 - root - INFO - connecting to 172.17.0.7 with username:root password:couchbase ssh_key: 2018-01-25 21:55:03,787 - root - INFO - Connected to 172.17.0.7 2018-01-25 21:55:04,264 - root - INFO - running command.raw on 172.17.0.7: zgrep "panic:" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l 2018-01-25 21:55:04,338 - root - INFO - command executed successfully 2018-01-25 21:55:04,339 - root - INFO - 0 2018-01-25 21:55:04,456 - root - INFO - connecting to 172.17.0.7 with username:root password:couchbase ssh_key: 2018-01-25 21:55:04,588 - root - INFO - Connected to 172.17.0.7 2018-01-25 21:55:05,142 - root - INFO - running command.raw on 172.17.0.7: zgrep "non-recoverable error from xmem client. response status=KEY_ENOENT" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l 2018-01-25 21:55:05,214 - root - INFO - command executed successfully 2018-01-25 21:55:05,215 - root - INFO - 0 2018-01-25 21:55:05,219 - root - INFO - connecting to 172.17.0.5 with username:root password:couchbase ssh_key: 2018-01-25 21:55:05,323 - root - INFO - Connected to 172.17.0.5 2018-01-25 21:55:05,842 - root - INFO - running command.raw on 172.17.0.5: zgrep "panic:" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l 2018-01-25 21:55:05,919 - root - INFO - command executed successfully 2018-01-25 21:55:05,919 - root - INFO - 0 2018-01-25 21:55:05,922 - root - INFO - connecting to 172.17.0.5 with username:root password:couchbase ssh_key: 2018-01-25 21:55:06,054 - root - INFO - Connected to 172.17.0.5 2018-01-25 21:55:06,369 - root - INFO - running command.raw on 172.17.0.5: zgrep "non-recoverable error from xmem client. response status=KEY_ENOENT" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l 2018-01-25 21:55:06,494 - root - INFO - command executed successfully 2018-01-25 21:55:06,496 - root - INFO - 0 2018-01-25 21:55:06,499 - root - INFO - {'172.17.0.8': {'non-recoverable error from xmem client. response status=KEY_ENOENT': 0, 'panic:': 0}, '172.17.0.6': {'non-recoverable error from xmem client. response status=KEY_ENOENT': 0, 'panic:': 0}, '172.17.0.7': {'non-recoverable error from xmem client. response status=KEY_ENOENT': 0, 'panic:': 0}, '172.17.0.5': {'non-recoverable error from xmem client. response status=KEY_ENOENT': 0, 'panic:': 0}} 2018-01-25 21:55:06,551 - root - INFO - /diag/eval status on 172.17.0.8:8091: True content: "/opt/couchbase/var/lib/couchbase/logs" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))). 2018-01-25 21:55:06,553 - root - INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: 2018-01-25 21:55:06,686 - root - INFO - Connected to 172.17.0.8 2018-01-25 21:55:07,073 - root - INFO - running command.raw on 172.17.0.8: zgrep "Try to fix Pipeline" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l 2018-01-25 21:55:07,148 - root - INFO - command executed successfully 2018-01-25 21:55:07,148 - root - INFO - 0 2018-01-25 21:55:07,266 - root - INFO - connecting to 172.17.0.6 with username:root password:couchbase ssh_key: 2018-01-25 21:55:07,403 - root - INFO - Connected to 172.17.0.6 2018-01-25 21:55:07,803 - root - INFO - running command.raw on 172.17.0.6: zgrep "Try to fix Pipeline" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l 2018-01-25 21:55:07,893 - root - INFO - command executed successfully 2018-01-25 21:55:07,894 - root - INFO - 0 2018-01-25 21:55:08,012 - root - INFO - connecting to 172.17.0.7 with username:root password:couchbase ssh_key: 2018-01-25 21:55:08,156 - root - INFO - Connected to 172.17.0.7 2018-01-25 21:55:08,618 - root - INFO - running command.raw on 172.17.0.7: zgrep "Try to fix Pipeline" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l 2018-01-25 21:55:08,701 - root - INFO - command executed successfully 2018-01-25 21:55:08,702 - root - INFO - 0 2018-01-25 21:55:08,704 - root - INFO - connecting to 172.17.0.5 with username:root password:couchbase ssh_key: 2018-01-25 21:55:08,850 - root - INFO - Connected to 172.17.0.5 2018-01-25 21:55:09,311 - root - INFO - running command.raw on 172.17.0.5: zgrep "Try to fix Pipeline" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l 2018-01-25 21:55:09,389 - root - INFO - command executed successfully 2018-01-25 21:55:09,389 - root - INFO - 0 2018-01-25 21:55:09,505 - root - INFO - {'172.17.0.8': 0, '172.17.0.6': 0, '172.17.0.7': 0, '172.17.0.5': 0} 2018-01-25 21:55:09,506 - root - INFO - ==== XDCRNewbasetests setup is started for test #30 load_with_ops ==== 2018-01-25 21:55:09,506 - root - INFO - Initializing all clusters... 2018-01-25 21:55:09,506 - root - INFO - ==== XDCRNewbasetests setup is finished for test #30 load_with_ops ==== load_with_ops (xdcr.uniXDCR.unidirectional) ... ERROR ====================================================================== ERROR: load_with_ops (xdcr.uniXDCR.unidirectional) ---------------------------------------------------------------------- Traceback (most recent call last): File "pytests/xdcr/uniXDCR.py", line 19, in setUp self.src_cluster = self.get_cb_cluster_by_name('C1') File "pytests/xdcr/xdcrnewbasetests.py", line 2791, in get_cb_cluster_by_name raise XDCRException("Couchbase Cluster with name: %s not exist" % name) XDCRException: Couchbase Cluster with name: C1 not exist ---------------------------------------------------------------------- Ran 1 test in 9.598s FAILED (errors=1) summary so far suite recreatebuckettests.RecreateMembaseBuckets , pass 0 , fail 1 failures so far... recreatebuckettests.RecreateMembaseBuckets.test_default_moxi summary so far suite deletebuckettests.DeleteMembaseBuckets , pass 1 , fail 0 summary so far suite createbuckettests.CreateMembaseBucketsTests , pass 1 , fail 4 failures so far... createbuckettests.CreateMembaseBucketsTests.test_default_moxi createbuckettests.CreateMembaseBucketsTests.test_default_on_non_default_port createbuckettests.CreateMembaseBucketsTests.test_non_default_case_sensitive_different_port createbuckettests.CreateMembaseBucketsTests.test_two_replica summary so far suite setgettests.MembaseBucket , pass 1 , fail 0 summary so far suite expirytests.ExpiryTests , pass 1 , fail 0 summary so far suite memcapable.GetlTests , pass 2 , fail 0 summary so far suite memorysanitytests.MemorySanity , pass 1 , fail 0 summary so far suite drainratetests.DrainRateTests , pass 1 , fail 0 summary so far suite view.viewquerytests.ViewQueryTests , pass 1 , fail 0 summary so far suite view.createdeleteview.CreateDeleteViewTests , pass 1 , fail 0 summary so far suite rebalance.rebalancein.RebalanceInTests , pass 0 , fail 1 failures so far... rebalance.rebalancein.RebalanceInTests.rebalance_in_with_ops summary so far suite rebalance.rebalanceout.RebalanceOutTests , pass 0 , fail 1 failures so far... rebalance.rebalanceout.RebalanceOutTests.rebalance_out_with_ops summary so far suite failover.failovertests.FailoverTests , pass 0 , fail 1 failures so far... failover.failovertests.FailoverTests.test_failover_normal summary so far suite CCCP.CCCP , pass 0 , fail 2 failures so far... CCCP.CCCP.test_get_config_client CCCP.CCCP.test_not_my_vbucket_config summary so far suite flush.bucketflush.BucketFlushTests , pass 0 , fail 1 failures so far... flush.bucketflush.BucketFlushTests.bucketflush_with_data_ops_moxi summary so far suite security.audittest.auditTest , pass 0 , fail 1 failures so far... security.audittest.auditTest.test_bucketEvents summary so far suite tuqquery.tuq_precedence.PrecedenceTests , pass 0 , fail 2 failures so far... tuqquery.tuq_precedence.PrecedenceTests.test_case_and_like tuqquery.tuq_precedence.PrecedenceTests.test_case_and_like summary so far suite tuqquery.tuq_index.QueriesViewsTests , pass 0 , fail 3 failures so far... tuqquery.tuq_index.QueriesViewsTests.test_primary_create_delete_index tuqquery.tuq_index.QueriesViewsTests.test_primary_create_delete_index tuqquery.tuq_index.QueriesViewsTests.test_explain_index_attr summary so far suite tuqquery.tuq_dml.DMLQueryTests , pass 0 , fail 1 failures so far... tuqquery.tuq_dml.DMLQueryTests.test_sanity summary so far suite 2i.indexscans_2i.SecondaryIndexingScanTests , pass 0 , fail 1 failures so far... 2i.indexscans_2i.SecondaryIndexingScanTests.test_multi_create_query_explain_drop_index summary so far suite xdcr.uniXDCR.unidirectional , pass 0 , fail 1 failures so far... xdcr.uniXDCR.unidirectional.load_with_ops testrunner logs, diags and results are available under /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_30 Logs will be stored at /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_31 ./testrunner -i ./4node.ini -t xdcr.filterXDCR.XDCRFilterTests.test_xdcr_with_filter,items=1000,rdirection=unidirection,ctopology=chain,default@C1=filter_expression:C1-key-1 Test Input params: {'items': '1000', 'case_number': 31, 'conf_file': './4node.conf', 'num_nodes': 4, 'cluster_name': '4node', 'ctopology': 'chain', 'rdirection': 'unidirection', 'ini': './4node.ini', 'default@C1': 'filter_expression:C1-key-1', 'logs_folder': '/testrunner/logs/testrunner-18-Jan-25_21-16-28/test_31', 'spec': '4node'} [2018-01-25 21:55:09,525] - [rest_client:1441] INFO - /diag/eval status on 172.17.0.8:8091: True content: "/opt/couchbase/var/lib/couchbase/logs" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))). [2018-01-25 21:55:09,527] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 21:55:09,658] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 21:55:10,095] - [remote_util:2995] INFO - running command.raw on 172.17.0.8: zgrep "panic:" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l [2018-01-25 21:55:10,170] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:55:10,171] - [xdcrnewbasetests:428] INFO - 0 [2018-01-25 21:55:10,287] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 21:55:10,419] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 21:55:10,874] - [remote_util:2995] INFO - running command.raw on 172.17.0.8: zgrep "non-recoverable error from xmem client. response status=KEY_ENOENT" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l [2018-01-25 21:55:10,963] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:55:10,964] - [xdcrnewbasetests:428] INFO - 0 [2018-01-25 21:55:10,969] - [remote_util:232] INFO - connecting to 172.17.0.6 with username:root password:couchbase ssh_key: [2018-01-25 21:55:11,135] - [remote_util:266] INFO - Connected to 172.17.0.6 [2018-01-25 21:55:11,693] - [remote_util:2995] INFO - running command.raw on 172.17.0.6: zgrep "panic:" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l [2018-01-25 21:55:11,773] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:55:11,773] - [xdcrnewbasetests:428] INFO - 0 [2018-01-25 21:55:11,894] - [remote_util:232] INFO - connecting to 172.17.0.6 with username:root password:couchbase ssh_key: [2018-01-25 21:55:12,032] - [remote_util:266] INFO - Connected to 172.17.0.6 [2018-01-25 21:55:12,504] - [remote_util:2995] INFO - running command.raw on 172.17.0.6: zgrep "non-recoverable error from xmem client. response status=KEY_ENOENT" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l [2018-01-25 21:55:12,612] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:55:12,613] - [xdcrnewbasetests:428] INFO - 0 [2018-01-25 21:55:12,730] - [remote_util:232] INFO - connecting to 172.17.0.7 with username:root password:couchbase ssh_key: [2018-01-25 21:55:12,831] - [remote_util:266] INFO - Connected to 172.17.0.7 [2018-01-25 21:55:13,414] - [remote_util:2995] INFO - running command.raw on 172.17.0.7: zgrep "panic:" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l [2018-01-25 21:55:13,498] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:55:13,500] - [xdcrnewbasetests:428] INFO - 0 [2018-01-25 21:55:13,616] - [remote_util:232] INFO - connecting to 172.17.0.7 with username:root password:couchbase ssh_key: [2018-01-25 21:55:13,748] - [remote_util:266] INFO - Connected to 172.17.0.7 [2018-01-25 21:55:15,072] - [remote_util:2995] INFO - running command.raw on 172.17.0.7: zgrep "non-recoverable error from xmem client. response status=KEY_ENOENT" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l [2018-01-25 21:55:15,171] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:55:15,172] - [xdcrnewbasetests:428] INFO - 0 [2018-01-25 21:55:15,291] - [remote_util:232] INFO - connecting to 172.17.0.5 with username:root password:couchbase ssh_key: [2018-01-25 21:55:15,437] - [remote_util:266] INFO - Connected to 172.17.0.5 [2018-01-25 21:55:16,067] - [remote_util:2995] INFO - running command.raw on 172.17.0.5: zgrep "panic:" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l [2018-01-25 21:55:16,144] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:55:16,145] - [xdcrnewbasetests:428] INFO - 0 [2018-01-25 21:55:16,261] - [remote_util:232] INFO - connecting to 172.17.0.5 with username:root password:couchbase ssh_key: [2018-01-25 21:55:16,343] - [remote_util:266] INFO - Connected to 172.17.0.5 [2018-01-25 21:55:16,845] - [remote_util:2995] INFO - running command.raw on 172.17.0.5: zgrep "non-recoverable error from xmem client. response status=KEY_ENOENT" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l [2018-01-25 21:55:16,921] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:55:16,921] - [xdcrnewbasetests:428] INFO - 0 [2018-01-25 21:55:17,037] - [xdcrnewbasetests:2705] INFO - {'172.17.0.8': {'non-recoverable error from xmem client. response status=KEY_ENOENT': 0, 'panic:': 0}, '172.17.0.6': {'non-recoverable error from xmem client. response status=KEY_ENOENT': 0, 'panic:': 0}, '172.17.0.7': {'non-recoverable error from xmem client. response status=KEY_ENOENT': 0, 'panic:': 0}, '172.17.0.5': {'non-recoverable error from xmem client. response status=KEY_ENOENT': 0, 'panic:': 0}} [2018-01-25 21:55:17,047] - [rest_client:1441] INFO - /diag/eval status on 172.17.0.8:8091: True content: "/opt/couchbase/var/lib/couchbase/logs" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))). [2018-01-25 21:55:17,048] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 21:55:17,130] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 21:55:17,595] - [remote_util:2995] INFO - running command.raw on 172.17.0.8: zgrep "Try to fix Pipeline" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l [2018-01-25 21:55:17,675] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:55:17,676] - [xdcrnewbasetests:428] INFO - 0 [2018-01-25 21:55:17,794] - [remote_util:232] INFO - connecting to 172.17.0.6 with username:root password:couchbase ssh_key: [2018-01-25 21:55:17,894] - [remote_util:266] INFO - Connected to 172.17.0.6 [2018-01-25 21:55:18,281] - [remote_util:2995] INFO - running command.raw on 172.17.0.6: zgrep "Try to fix Pipeline" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l [2018-01-25 21:55:18,355] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:55:18,355] - [xdcrnewbasetests:428] INFO - 0 [2018-01-25 21:55:18,473] - [remote_util:232] INFO - connecting to 172.17.0.7 with username:root password:couchbase ssh_key: [2018-01-25 21:55:18,609] - [remote_util:266] INFO - Connected to 172.17.0.7 [2018-01-25 21:55:19,042] - [remote_util:2995] INFO - running command.raw on 172.17.0.7: zgrep "Try to fix Pipeline" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l [2018-01-25 21:55:19,118] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:55:19,119] - [xdcrnewbasetests:428] INFO - 0 [2018-01-25 21:55:19,121] - [remote_util:232] INFO - connecting to 172.17.0.5 with username:root password:couchbase ssh_key: [2018-01-25 21:55:19,252] - [remote_util:266] INFO - Connected to 172.17.0.5 [2018-01-25 21:55:19,609] - [remote_util:2995] INFO - running command.raw on 172.17.0.5: zgrep "Try to fix Pipeline" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l [2018-01-25 21:55:19,663] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:55:19,663] - [xdcrnewbasetests:428] INFO - 0 [2018-01-25 21:55:19,665] - [xdcrnewbasetests:2723] INFO - {'172.17.0.8': 0, '172.17.0.6': 0, '172.17.0.7': 0, '172.17.0.5': 0} [2018-01-25 21:55:19,665] - [xdcrnewbasetests:2486] INFO - ==== XDCRNewbasetests setup is started for test #31 test_xdcr_with_filter ==== [2018-01-25 21:55:19,666] - [xdcrnewbasetests:2733] INFO - Initializing all clusters... [2018-01-25 21:55:19,666] - [xdcrnewbasetests:2492] INFO - ==== XDCRNewbasetests setup is finished for test #31 test_xdcr_with_filter ==== [2018-01-25 21:55:19,666] - [xdcrnewbasetests:3468] INFO - sleep for 10 secs. ... [2018-01-25 21:55:29,685] - [rest_client:1441] INFO - /diag/eval status on 172.17.0.8:8091: True content: "/opt/couchbase/var/lib/couchbase/logs" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))). [2018-01-25 21:55:29,686] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 21:55:29,768] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 21:55:30,203] - [remote_util:2995] INFO - running command.raw on 172.17.0.8: zgrep "Try to fix Pipeline" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* [2018-01-25 21:55:30,276] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:55:30,277] - [remote_util:2995] INFO - running command.raw on 172.17.0.8: zgrep "Try to fix Pipeline" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l [2018-01-25 21:55:30,352] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:55:30,353] - [xdcrnewbasetests:428] INFO - 0 [2018-01-25 21:55:30,468] - [xdcrnewbasetests:3382] INFO - Initial replication restart count on 172.17.0.8 :0, now :0 [2018-01-25 21:55:30,469] - [remote_util:232] INFO - connecting to 172.17.0.6 with username:root password:couchbase ssh_key: [2018-01-25 21:55:30,601] - [remote_util:266] INFO - Connected to 172.17.0.6 [2018-01-25 21:55:31,005] - [remote_util:2995] INFO - running command.raw on 172.17.0.6: zgrep "Try to fix Pipeline" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* [2018-01-25 21:55:31,077] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:55:31,078] - [remote_util:2995] INFO - running command.raw on 172.17.0.6: zgrep "Try to fix Pipeline" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l [2018-01-25 21:55:31,155] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:55:31,155] - [xdcrnewbasetests:428] INFO - 0 [2018-01-25 21:55:31,159] - [xdcrnewbasetests:3382] INFO - Initial replication restart count on 172.17.0.6 :0, now :0 [2018-01-25 21:55:31,161] - [remote_util:232] INFO - connecting to 172.17.0.7 with username:root password:couchbase ssh_key: [2018-01-25 21:55:31,294] - [remote_util:266] INFO - Connected to 172.17.0.7 [2018-01-25 21:55:31,783] - [remote_util:2995] INFO - running command.raw on 172.17.0.7: zgrep "Try to fix Pipeline" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* [2018-01-25 21:55:31,857] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:55:31,858] - [remote_util:2995] INFO - running command.raw on 172.17.0.7: zgrep "Try to fix Pipeline" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l [2018-01-25 21:55:31,936] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:55:31,937] - [xdcrnewbasetests:428] INFO - 0 [2018-01-25 21:55:32,054] - [xdcrnewbasetests:3382] INFO - Initial replication restart count on 172.17.0.7 :0, now :0 [2018-01-25 21:55:32,055] - [remote_util:232] INFO - connecting to 172.17.0.5 with username:root password:couchbase ssh_key: [2018-01-25 21:55:32,137] - [remote_util:266] INFO - Connected to 172.17.0.5 [2018-01-25 21:55:32,587] - [remote_util:2995] INFO - running command.raw on 172.17.0.5: zgrep "Try to fix Pipeline" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* [2018-01-25 21:55:32,663] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:55:32,664] - [remote_util:2995] INFO - running command.raw on 172.17.0.5: zgrep "Try to fix Pipeline" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l [2018-01-25 21:55:32,746] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:55:32,746] - [xdcrnewbasetests:428] INFO - 0 [2018-01-25 21:55:32,861] - [xdcrnewbasetests:3382] INFO - Initial replication restart count on 172.17.0.5 :0, now :0 [2018-01-25 21:55:32,873] - [rest_client:1441] INFO - /diag/eval status on 172.17.0.8:8091: True content: "/opt/couchbase/var/lib/couchbase/logs" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))). [2018-01-25 21:55:32,874] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 21:55:33,005] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 21:55:33,424] - [remote_util:2995] INFO - running command.raw on 172.17.0.8: zgrep "panic:" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l [2018-01-25 21:55:33,501] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:55:33,501] - [xdcrnewbasetests:428] INFO - 0 [2018-01-25 21:55:33,503] - [xdcrnewbasetests:3352] INFO - Initial panic: count on 172.17.0.8 :0, now :0 [2018-01-25 21:55:33,529] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 21:55:33,627] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 21:55:34,189] - [remote_util:2995] INFO - running command.raw on 172.17.0.8: zgrep "non-recoverable error from xmem client. response status=KEY_ENOENT" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l [2018-01-25 21:55:34,267] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:55:34,267] - [xdcrnewbasetests:428] INFO - 0 [2018-01-25 21:55:34,383] - [xdcrnewbasetests:3352] INFO - Initial non-recoverable error from xmem client. response status=KEY_ENOENT count on 172.17.0.8 :0, now :0 [2018-01-25 21:55:34,385] - [remote_util:232] INFO - connecting to 172.17.0.6 with username:root password:couchbase ssh_key: [2018-01-25 21:55:34,520] - [remote_util:266] INFO - Connected to 172.17.0.6 [2018-01-25 21:55:34,970] - [remote_util:2995] INFO - running command.raw on 172.17.0.6: zgrep "panic:" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l [2018-01-25 21:55:35,046] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:55:35,047] - [xdcrnewbasetests:428] INFO - 0 [2018-01-25 21:55:35,162] - [xdcrnewbasetests:3352] INFO - Initial panic: count on 172.17.0.6 :0, now :0 [2018-01-25 21:55:35,163] - [remote_util:232] INFO - connecting to 172.17.0.6 with username:root password:couchbase ssh_key: [2018-01-25 21:55:35,294] - [remote_util:266] INFO - Connected to 172.17.0.6 [2018-01-25 21:55:35,684] - [remote_util:2995] INFO - running command.raw on 172.17.0.6: zgrep "non-recoverable error from xmem client. response status=KEY_ENOENT" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l [2018-01-25 21:55:35,762] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:55:35,762] - [xdcrnewbasetests:428] INFO - 0 [2018-01-25 21:55:35,880] - [xdcrnewbasetests:3352] INFO - Initial non-recoverable error from xmem client. response status=KEY_ENOENT count on 172.17.0.6 :0, now :0 [2018-01-25 21:55:35,881] - [remote_util:232] INFO - connecting to 172.17.0.7 with username:root password:couchbase ssh_key: [2018-01-25 21:55:36,015] - [remote_util:266] INFO - Connected to 172.17.0.7 [2018-01-25 21:55:36,345] - [remote_util:2995] INFO - running command.raw on 172.17.0.7: zgrep "panic:" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l [2018-01-25 21:55:36,428] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:55:36,429] - [xdcrnewbasetests:428] INFO - 0 [2018-01-25 21:55:36,547] - [xdcrnewbasetests:3352] INFO - Initial panic: count on 172.17.0.7 :0, now :0 [2018-01-25 21:55:36,549] - [remote_util:232] INFO - connecting to 172.17.0.7 with username:root password:couchbase ssh_key: [2018-01-25 21:55:36,682] - [remote_util:266] INFO - Connected to 172.17.0.7 [2018-01-25 21:55:37,178] - [remote_util:2995] INFO - running command.raw on 172.17.0.7: zgrep "non-recoverable error from xmem client. response status=KEY_ENOENT" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l [2018-01-25 21:55:37,255] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:55:37,255] - [xdcrnewbasetests:428] INFO - 0 [2018-01-25 21:55:37,257] - [xdcrnewbasetests:3352] INFO - Initial non-recoverable error from xmem client. response status=KEY_ENOENT count on 172.17.0.7 :0, now :0 [2018-01-25 21:55:37,258] - [remote_util:232] INFO - connecting to 172.17.0.5 with username:root password:couchbase ssh_key: [2018-01-25 21:55:37,342] - [remote_util:266] INFO - Connected to 172.17.0.5 [2018-01-25 21:55:37,919] - [remote_util:2995] INFO - running command.raw on 172.17.0.5: zgrep "panic:" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l [2018-01-25 21:55:37,996] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:55:37,997] - [xdcrnewbasetests:428] INFO - 0 [2018-01-25 21:55:38,115] - [xdcrnewbasetests:3352] INFO - Initial panic: count on 172.17.0.5 :0, now :0 [2018-01-25 21:55:38,117] - [remote_util:232] INFO - connecting to 172.17.0.5 with username:root password:couchbase ssh_key: [2018-01-25 21:55:38,200] - [remote_util:266] INFO - Connected to 172.17.0.5 [2018-01-25 21:55:38,619] - [remote_util:2995] INFO - running command.raw on 172.17.0.5: zgrep "non-recoverable error from xmem client. response status=KEY_ENOENT" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l [2018-01-25 21:55:38,705] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:55:38,705] - [xdcrnewbasetests:428] INFO - 0 [2018-01-25 21:55:38,821] - [xdcrnewbasetests:3352] INFO - Initial non-recoverable error from xmem client. response status=KEY_ENOENT count on 172.17.0.5 :0, now :0 [2018-01-25 21:55:38,821] - [xdcrnewbasetests:2544] INFO - ==== XDCRNewbasetests cleanup is started for test #31 test_xdcr_with_filter ==== [2018-01-25 21:55:38,821] - [xdcrnewbasetests:2549] INFO - ==== XDCRNewbasetests cleanup is finished for test #31 test_xdcr_with_filter === test_xdcr_with_filter (xdcr.filterXDCR.XDCRFilterTests) ... ok ---------------------------------------------------------------------- Ran 1 test in 29.304s OK Cluster instance shutdown with force summary so far suite recreatebuckettests.RecreateMembaseBuckets , pass 0 , fail 1 failures so far... recreatebuckettests.RecreateMembaseBuckets.test_default_moxi summary so far suite deletebuckettests.DeleteMembaseBuckets , pass 1 , fail 0 summary so far suite createbuckettests.CreateMembaseBucketsTests , pass 1 , fail 4 failures so far... createbuckettests.CreateMembaseBucketsTests.test_default_moxi createbuckettests.CreateMembaseBucketsTests.test_default_on_non_default_port createbuckettests.CreateMembaseBucketsTests.test_non_default_case_sensitive_different_port createbuckettests.CreateMembaseBucketsTests.test_two_replica summary so far suite setgettests.MembaseBucket , pass 1 , fail 0 summary so far suite expirytests.ExpiryTests , pass 1 , fail 0 summary so far suite memcapable.GetlTests , pass 2 , fail 0 summary so far suite memorysanitytests.MemorySanity , pass 1 , fail 0 summary so far suite drainratetests.DrainRateTests , pass 1 , fail 0 summary so far suite view.viewquerytests.ViewQueryTests , pass 1 , fail 0 summary so far suite view.createdeleteview.CreateDeleteViewTests , pass 1 , fail 0 summary so far suite rebalance.rebalancein.RebalanceInTests , pass 0 , fail 1 failures so far... rebalance.rebalancein.RebalanceInTests.rebalance_in_with_ops summary so far suite rebalance.rebalanceout.RebalanceOutTests , pass 0 , fail 1 failures so far... rebalance.rebalanceout.RebalanceOutTests.rebalance_out_with_ops summary so far suite failover.failovertests.FailoverTests , pass 0 , fail 1 failures so far... failover.failovertests.FailoverTests.test_failover_normal summary so far suite CCCP.CCCP , pass 0 , fail 2 failures so far... CCCP.CCCP.test_get_config_client CCCP.CCCP.test_not_my_vbucket_config summary so far suite flush.bucketflush.BucketFlushTests , pass 0 , fail 1 failures so far... flush.bucketflush.BucketFlushTests.bucketflush_with_data_ops_moxi summary so far suite security.audittest.auditTest , pass 0 , fail 1 failures so far... security.audittest.auditTest.test_bucketEvents summary so far suite tuqquery.tuq_precedence.PrecedenceTests , pass 0 , fail 2 failures so far... tuqquery.tuq_precedence.PrecedenceTests.test_case_and_like tuqquery.tuq_precedence.PrecedenceTests.test_case_and_like summary so far suite tuqquery.tuq_index.QueriesViewsTests , pass 0 , fail 3 failures so far... tuqquery.tuq_index.QueriesViewsTests.test_primary_create_delete_index tuqquery.tuq_index.QueriesViewsTests.test_primary_create_delete_index tuqquery.tuq_index.QueriesViewsTests.test_explain_index_attr summary so far suite tuqquery.tuq_dml.DMLQueryTests , pass 0 , fail 1 failures so far... tuqquery.tuq_dml.DMLQueryTests.test_sanity summary so far suite 2i.indexscans_2i.SecondaryIndexingScanTests , pass 0 , fail 1 failures so far... 2i.indexscans_2i.SecondaryIndexingScanTests.test_multi_create_query_explain_drop_index summary so far suite xdcr.uniXDCR.unidirectional , pass 0 , fail 1 failures so far... xdcr.uniXDCR.unidirectional.load_with_ops summary so far suite xdcr.filterXDCR.XDCRFilterTests , pass 1 , fail 0 testrunner logs, diags and results are available under /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_31 Logs will be stored at /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_32 ./testrunner -i ./4node.ini -t xdcr.biXDCR.bidirectional.load_with_async_ops,replicas=1,items=1000,ctopology=chain,rdirection=bidirection,update=C1-C2,delete=C1-C2 Test Input params: {'replicas': '1', 'items': '1000', 'update': 'C1-C2', 'conf_file': './4node.conf', 'num_nodes': 4, 'cluster_name': '4node', 'ctopology': 'chain', 'rdirection': 'bidirection', 'ini': './4node.ini', 'case_number': 32, 'logs_folder': '/testrunner/logs/testrunner-18-Jan-25_21-16-28/test_32', 'spec': '4node', 'delete': 'C1-C2'} [2018-01-25 21:55:38,864] - [rest_client:1441] INFO - /diag/eval status on 172.17.0.8:8091: True content: "/opt/couchbase/var/lib/couchbase/logs" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))). [2018-01-25 21:55:38,865] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 21:55:38,996] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 21:55:39,349] - [remote_util:2995] INFO - running command.raw on 172.17.0.8: zgrep "panic:" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l [2018-01-25 21:55:39,424] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:55:39,425] - [xdcrnewbasetests:428] INFO - 0 [2018-01-25 21:55:39,542] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 21:55:39,629] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 21:55:40,075] - [remote_util:2995] INFO - running command.raw on 172.17.0.8: zgrep "non-recoverable error from xmem client. response status=KEY_ENOENT" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l [2018-01-25 21:55:40,176] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:55:40,181] - [xdcrnewbasetests:428] INFO - 0 [2018-01-25 21:55:40,184] - [remote_util:232] INFO - connecting to 172.17.0.6 with username:root password:couchbase ssh_key: [2018-01-25 21:55:40,324] - [remote_util:266] INFO - Connected to 172.17.0.6 [2018-01-25 21:55:40,874] - [remote_util:2995] INFO - running command.raw on 172.17.0.6: zgrep "panic:" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l [2018-01-25 21:55:40,972] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:55:40,974] - [xdcrnewbasetests:428] INFO - 0 [2018-01-25 21:55:41,112] - [remote_util:232] INFO - connecting to 172.17.0.6 with username:root password:couchbase ssh_key: [2018-01-25 21:55:41,283] - [remote_util:266] INFO - Connected to 172.17.0.6 [2018-01-25 21:55:41,750] - [remote_util:2995] INFO - running command.raw on 172.17.0.6: zgrep "non-recoverable error from xmem client. response status=KEY_ENOENT" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l [2018-01-25 21:55:41,776] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:55:41,776] - [xdcrnewbasetests:428] INFO - 0 [2018-01-25 21:55:41,779] - [remote_util:232] INFO - connecting to 172.17.0.7 with username:root password:couchbase ssh_key: [2018-01-25 21:55:41,910] - [remote_util:266] INFO - Connected to 172.17.0.7 [2018-01-25 21:55:42,272] - [remote_util:2995] INFO - running command.raw on 172.17.0.7: zgrep "panic:" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l [2018-01-25 21:55:42,355] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:55:42,355] - [xdcrnewbasetests:428] INFO - 0 [2018-01-25 21:55:42,361] - [remote_util:232] INFO - connecting to 172.17.0.7 with username:root password:couchbase ssh_key: [2018-01-25 21:55:42,462] - [remote_util:266] INFO - Connected to 172.17.0.7 [2018-01-25 21:55:43,145] - [remote_util:2995] INFO - running command.raw on 172.17.0.7: zgrep "non-recoverable error from xmem client. response status=KEY_ENOENT" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l [2018-01-25 21:55:43,250] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:55:43,251] - [xdcrnewbasetests:428] INFO - 0 [2018-01-25 21:55:43,368] - [remote_util:232] INFO - connecting to 172.17.0.5 with username:root password:couchbase ssh_key: [2018-01-25 21:55:43,502] - [remote_util:266] INFO - Connected to 172.17.0.5 [2018-01-25 21:55:44,051] - [remote_util:2995] INFO - running command.raw on 172.17.0.5: zgrep "panic:" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l [2018-01-25 21:55:44,135] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:55:44,135] - [xdcrnewbasetests:428] INFO - 0 [2018-01-25 21:55:44,138] - [remote_util:232] INFO - connecting to 172.17.0.5 with username:root password:couchbase ssh_key: [2018-01-25 21:55:44,241] - [remote_util:266] INFO - Connected to 172.17.0.5 [2018-01-25 21:55:44,707] - [remote_util:2995] INFO - running command.raw on 172.17.0.5: zgrep "non-recoverable error from xmem client. response status=KEY_ENOENT" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l [2018-01-25 21:55:44,789] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:55:44,789] - [xdcrnewbasetests:428] INFO - 0 [2018-01-25 21:55:44,793] - [xdcrnewbasetests:2705] INFO - {'172.17.0.8': {'non-recoverable error from xmem client. response status=KEY_ENOENT': 0, 'panic:': 0}, '172.17.0.6': {'non-recoverable error from xmem client. response status=KEY_ENOENT': 0, 'panic:': 0}, '172.17.0.7': {'non-recoverable error from xmem client. response status=KEY_ENOENT': 0, 'panic:': 0}, '172.17.0.5': {'non-recoverable error from xmem client. response status=KEY_ENOENT': 0, 'panic:': 0}} [2018-01-25 21:55:44,805] - [rest_client:1441] INFO - /diag/eval status on 172.17.0.8:8091: True content: "/opt/couchbase/var/lib/couchbase/logs" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))). [2018-01-25 21:55:44,806] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 21:55:44,944] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 21:55:45,438] - [remote_util:2995] INFO - running command.raw on 172.17.0.8: zgrep "Try to fix Pipeline" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l [2018-01-25 21:55:45,525] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:55:45,527] - [xdcrnewbasetests:428] INFO - 0 [2018-01-25 21:55:45,644] - [remote_util:232] INFO - connecting to 172.17.0.6 with username:root password:couchbase ssh_key: [2018-01-25 21:55:45,777] - [remote_util:266] INFO - Connected to 172.17.0.6 [2018-01-25 21:55:46,215] - [remote_util:2995] INFO - running command.raw on 172.17.0.6: zgrep "Try to fix Pipeline" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l [2018-01-25 21:55:46,302] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:55:46,303] - [xdcrnewbasetests:428] INFO - 0 [2018-01-25 21:55:46,307] - [remote_util:232] INFO - connecting to 172.17.0.7 with username:root password:couchbase ssh_key: [2018-01-25 21:55:46,388] - [remote_util:266] INFO - Connected to 172.17.0.7 [2018-01-25 21:55:46,841] - [remote_util:2995] INFO - running command.raw on 172.17.0.7: zgrep "Try to fix Pipeline" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l [2018-01-25 21:55:46,935] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:55:46,937] - [xdcrnewbasetests:428] INFO - 0 [2018-01-25 21:55:47,074] - [remote_util:232] INFO - connecting to 172.17.0.5 with username:root password:couchbase ssh_key: [2018-01-25 21:55:47,218] - [remote_util:266] INFO - Connected to 172.17.0.5 [2018-01-25 21:55:47,613] - [remote_util:2995] INFO - running command.raw on 172.17.0.5: zgrep "Try to fix Pipeline" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l [2018-01-25 21:55:47,688] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:55:47,688] - [xdcrnewbasetests:428] INFO - 0 [2018-01-25 21:55:47,690] - [xdcrnewbasetests:2723] INFO - {'172.17.0.8': 0, '172.17.0.6': 0, '172.17.0.7': 0, '172.17.0.5': 0} [2018-01-25 21:55:47,691] - [xdcrnewbasetests:2486] INFO - ==== XDCRNewbasetests setup is started for test #32 load_with_async_ops ==== [2018-01-25 21:55:47,691] - [xdcrnewbasetests:2733] INFO - Initializing all clusters... [2018-01-25 21:55:47,691] - [xdcrnewbasetests:2492] INFO - ==== XDCRNewbasetests setup is finished for test #32 load_with_async_ops ==== load_with_async_ops (xdcr.biXDCR.bidirectional) ... ERROR ====================================================================== ERROR: load_with_async_ops (xdcr.biXDCR.bidirectional) ---------------------------------------------------------------------- Traceback (most recent call last): File "pytests/xdcr/biXDCR.py", line 17, in setUp self.src_cluster = self.get_cb_cluster_by_name('C1') File "pytests/xdcr/xdcrnewbasetests.py", line 2791, in get_cb_cluster_by_name raise XDCRException("Couchbase Cluster with name: %s not exist" % name) XDCRException: Couchbase Cluster with name: C1 not exist ---------------------------------------------------------------------- Ran 1 test in 8.840s FAILED (errors=1) summary so far suite recreatebuckettests.RecreateMembaseBuckets , pass 0 , fail 1 failures so far... recreatebuckettests.RecreateMembaseBuckets.test_default_moxi summary so far suite deletebuckettests.DeleteMembaseBuckets , pass 1 , fail 0 summary so far suite createbuckettests.CreateMembaseBucketsTests , pass 1 , fail 4 failures so far... createbuckettests.CreateMembaseBucketsTests.test_default_moxi createbuckettests.CreateMembaseBucketsTests.test_default_on_non_default_port createbuckettests.CreateMembaseBucketsTests.test_non_default_case_sensitive_different_port createbuckettests.CreateMembaseBucketsTests.test_two_replica summary so far suite setgettests.MembaseBucket , pass 1 , fail 0 summary so far suite expirytests.ExpiryTests , pass 1 , fail 0 summary so far suite memcapable.GetlTests , pass 2 , fail 0 summary so far suite memorysanitytests.MemorySanity , pass 1 , fail 0 summary so far suite drainratetests.DrainRateTests , pass 1 , fail 0 summary so far suite view.viewquerytests.ViewQueryTests , pass 1 , fail 0 summary so far suite view.createdeleteview.CreateDeleteViewTests , pass 1 , fail 0 summary so far suite rebalance.rebalancein.RebalanceInTests , pass 0 , fail 1 failures so far... rebalance.rebalancein.RebalanceInTests.rebalance_in_with_ops summary so far suite rebalance.rebalanceout.RebalanceOutTests , pass 0 , fail 1 failures so far... rebalance.rebalanceout.RebalanceOutTests.rebalance_out_with_ops summary so far suite failover.failovertests.FailoverTests , pass 0 , fail 1 failures so far... failover.failovertests.FailoverTests.test_failover_normal summary so far suite CCCP.CCCP , pass 0 , fail 2 failures so far... CCCP.CCCP.test_get_config_client CCCP.CCCP.test_not_my_vbucket_config summary so far suite flush.bucketflush.BucketFlushTests , pass 0 , fail 1 failures so far... flush.bucketflush.BucketFlushTests.bucketflush_with_data_ops_moxi summary so far suite security.audittest.auditTest , pass 0 , fail 1 failures so far... security.audittest.auditTest.test_bucketEvents summary so far suite tuqquery.tuq_precedence.PrecedenceTests , pass 0 , fail 2 failures so far... tuqquery.tuq_precedence.PrecedenceTests.test_case_and_like tuqquery.tuq_precedence.PrecedenceTests.test_case_and_like summary so far suite tuqquery.tuq_index.QueriesViewsTests , pass 0 , fail 3 failures so far... tuqquery.tuq_index.QueriesViewsTests.test_primary_create_delete_index tuqquery.tuq_index.QueriesViewsTests.test_primary_create_delete_index tuqquery.tuq_index.QueriesViewsTests.test_explain_index_attr summary so far suite tuqquery.tuq_dml.DMLQueryTests , pass 0 , fail 1 failures so far... tuqquery.tuq_dml.DMLQueryTests.test_sanity summary so far suite 2i.indexscans_2i.SecondaryIndexingScanTests , pass 0 , fail 1 failures so far... 2i.indexscans_2i.SecondaryIndexingScanTests.test_multi_create_query_explain_drop_index summary so far suite xdcr.uniXDCR.unidirectional , pass 0 , fail 1 failures so far... xdcr.uniXDCR.unidirectional.load_with_ops summary so far suite xdcr.filterXDCR.XDCRFilterTests , pass 1 , fail 0 summary so far suite xdcr.biXDCR.bidirectional , pass 0 , fail 1 failures so far... xdcr.biXDCR.bidirectional.load_with_async_ops testrunner logs, diags and results are available under /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_32 Logs will be stored at /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_33 ./testrunner -i ./4node.ini -t xdcr.filterXDCR.XDCRFilterTests.test_xdcr_with_filter,items=1000,pause=C1:C2,rdirection=bidirection,ctopology=chain,default@C1=filter_expression:C1-key-1,default@C2=filter_expression:C2-key-1,update=C1,delete=C1,demand_encryption=1 Test Input params: {'pause': 'C1:C2', 'demand_encryption': '1', 'items': '1000', 'case_number': 33, 'update': 'C1', 'conf_file': './4node.conf', 'num_nodes': 4, 'cluster_name': '4node', 'ctopology': 'chain', 'rdirection': 'bidirection', 'default@C2': 'filter_expression:C2-key-1', 'ini': './4node.ini', 'default@C1': 'filter_expression:C1-key-1', 'logs_folder': '/testrunner/logs/testrunner-18-Jan-25_21-16-28/test_33', 'spec': '4node', 'delete': 'C1'} [2018-01-25 21:55:47,715] - [rest_client:1441] INFO - /diag/eval status on 172.17.0.8:8091: True content: "/opt/couchbase/var/lib/couchbase/logs" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))). [2018-01-25 21:55:47,716] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 21:55:47,848] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 21:55:48,303] - [remote_util:2995] INFO - running command.raw on 172.17.0.8: zgrep "panic:" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l [2018-01-25 21:55:48,388] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:55:48,389] - [xdcrnewbasetests:428] INFO - 0 [2018-01-25 21:55:48,506] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 21:55:48,592] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 21:55:48,979] - [remote_util:2995] INFO - running command.raw on 172.17.0.8: zgrep "non-recoverable error from xmem client. response status=KEY_ENOENT" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l [2018-01-25 21:55:49,053] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:55:49,054] - [xdcrnewbasetests:428] INFO - 0 [2018-01-25 21:55:50,091] - [remote_util:232] INFO - connecting to 172.17.0.6 with username:root password:couchbase ssh_key: [2018-01-25 21:55:50,230] - [remote_util:266] INFO - Connected to 172.17.0.6 [2018-01-25 21:55:51,448] - [remote_util:2995] INFO - running command.raw on 172.17.0.6: zgrep "panic:" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l [2018-01-25 21:55:51,841] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:55:51,841] - [xdcrnewbasetests:428] INFO - 0 [2018-01-25 21:55:51,963] - [remote_util:232] INFO - connecting to 172.17.0.6 with username:root password:couchbase ssh_key: [2018-01-25 21:55:52,101] - [remote_util:266] INFO - Connected to 172.17.0.6 [2018-01-25 21:55:52,681] - [remote_util:2995] INFO - running command.raw on 172.17.0.6: zgrep "non-recoverable error from xmem client. response status=KEY_ENOENT" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l [2018-01-25 21:55:52,757] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:55:52,758] - [xdcrnewbasetests:428] INFO - 0 [2018-01-25 21:55:52,875] - [remote_util:232] INFO - connecting to 172.17.0.7 with username:root password:couchbase ssh_key: [2018-01-25 21:55:53,008] - [remote_util:266] INFO - Connected to 172.17.0.7 [2018-01-25 21:55:53,414] - [remote_util:2995] INFO - running command.raw on 172.17.0.7: zgrep "panic:" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l [2018-01-25 21:55:53,491] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:55:53,491] - [xdcrnewbasetests:428] INFO - 0 [2018-01-25 21:55:53,494] - [remote_util:232] INFO - connecting to 172.17.0.7 with username:root password:couchbase ssh_key: [2018-01-25 21:55:53,624] - [remote_util:266] INFO - Connected to 172.17.0.7 [2018-01-25 21:55:53,984] - [remote_util:2995] INFO - running command.raw on 172.17.0.7: zgrep "non-recoverable error from xmem client. response status=KEY_ENOENT" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l [2018-01-25 21:55:54,139] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:55:54,139] - [xdcrnewbasetests:428] INFO - 0 [2018-01-25 21:55:54,256] - [remote_util:232] INFO - connecting to 172.17.0.5 with username:root password:couchbase ssh_key: [2018-01-25 21:55:54,388] - [remote_util:266] INFO - Connected to 172.17.0.5 [2018-01-25 21:55:54,762] - [remote_util:2995] INFO - running command.raw on 172.17.0.5: zgrep "panic:" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l [2018-01-25 21:55:54,837] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:55:54,838] - [xdcrnewbasetests:428] INFO - 0 [2018-01-25 21:55:54,954] - [remote_util:232] INFO - connecting to 172.17.0.5 with username:root password:couchbase ssh_key: [2018-01-25 21:55:55,058] - [remote_util:266] INFO - Connected to 172.17.0.5 [2018-01-25 21:55:55,445] - [remote_util:2995] INFO - running command.raw on 172.17.0.5: zgrep "non-recoverable error from xmem client. response status=KEY_ENOENT" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l [2018-01-25 21:55:55,518] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:55:55,519] - [xdcrnewbasetests:428] INFO - 0 [2018-01-25 21:55:55,634] - [xdcrnewbasetests:2705] INFO - {'172.17.0.8': {'non-recoverable error from xmem client. response status=KEY_ENOENT': 0, 'panic:': 0}, '172.17.0.6': {'non-recoverable error from xmem client. response status=KEY_ENOENT': 0, 'panic:': 0}, '172.17.0.7': {'non-recoverable error from xmem client. response status=KEY_ENOENT': 0, 'panic:': 0}, '172.17.0.5': {'non-recoverable error from xmem client. response status=KEY_ENOENT': 0, 'panic:': 0}} [2018-01-25 21:55:55,647] - [rest_client:1441] INFO - /diag/eval status on 172.17.0.8:8091: True content: "/opt/couchbase/var/lib/couchbase/logs" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))). [2018-01-25 21:55:55,648] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 21:55:55,733] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 21:55:56,111] - [remote_util:2995] INFO - running command.raw on 172.17.0.8: zgrep "Try to fix Pipeline" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l [2018-01-25 21:55:56,197] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:55:56,199] - [xdcrnewbasetests:428] INFO - 0 [2018-01-25 21:55:56,206] - [remote_util:232] INFO - connecting to 172.17.0.6 with username:root password:couchbase ssh_key: [2018-01-25 21:55:56,413] - [remote_util:266] INFO - Connected to 172.17.0.6 [2018-01-25 21:55:57,240] - [remote_util:2995] INFO - running command.raw on 172.17.0.6: zgrep "Try to fix Pipeline" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l [2018-01-25 21:55:57,323] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:55:57,323] - [xdcrnewbasetests:428] INFO - 0 [2018-01-25 21:55:57,325] - [remote_util:232] INFO - connecting to 172.17.0.7 with username:root password:couchbase ssh_key: [2018-01-25 21:55:57,397] - [remote_util:266] INFO - Connected to 172.17.0.7 [2018-01-25 21:55:58,425] - [remote_util:2995] INFO - running command.raw on 172.17.0.7: zgrep "Try to fix Pipeline" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l [2018-01-25 21:55:58,503] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:55:58,504] - [xdcrnewbasetests:428] INFO - 0 [2018-01-25 21:55:58,621] - [remote_util:232] INFO - connecting to 172.17.0.5 with username:root password:couchbase ssh_key: [2018-01-25 21:55:58,799] - [remote_util:266] INFO - Connected to 172.17.0.5 [2018-01-25 21:55:59,203] - [remote_util:2995] INFO - running command.raw on 172.17.0.5: zgrep "Try to fix Pipeline" "/opt/couchbase/var/lib/couchbase/logs"/goxdcr.log* | wc -l [2018-01-25 21:55:59,280] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:55:59,280] - [xdcrnewbasetests:428] INFO - 0 [2018-01-25 21:55:59,982] - [xdcrnewbasetests:2723] INFO - {'172.17.0.8': 0, '172.17.0.6': 0, '172.17.0.7': 0, '172.17.0.5': 0} [2018-01-25 21:55:59,983] - [xdcrnewbasetests:2486] INFO - ==== XDCRNewbasetests setup is started for test #33 test_xdcr_with_filter ==== [2018-01-25 21:55:59,984] - [xdcrnewbasetests:2733] INFO - Initializing all clusters... [2018-01-25 21:55:59,984] - [xdcrnewbasetests:2492] INFO - ==== XDCRNewbasetests setup is finished for test #33 test_xdcr_with_filter ==== [2018-01-25 21:56:00,004] - [xdcrnewbasetests:3468] INFO - sleep for 10 secs. ... [2018-01-25 21:56:10,008] - [xdcrnewbasetests:2544] INFO - ==== XDCRNewbasetests cleanup is started for test #33 test_xdcr_with_filter ==== [2018-01-25 21:56:10,008] - [xdcrnewbasetests:2549] INFO - ==== XDCRNewbasetests cleanup is finished for test #33 test_xdcr_with_filter === test_xdcr_with_filter (xdcr.filterXDCR.XDCRFilterTests) ... ERROR ====================================================================== ERROR: test_xdcr_with_filter (xdcr.filterXDCR.XDCRFilterTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "pytests/xdcr/filterXDCR.py", line 41, in test_xdcr_with_filter for cluster in self.get_cluster_objects_for_input(pause): File "pytests/xdcr/filterXDCR.py", line 20, in get_cluster_objects_for_input clusters.append(self.get_cb_cluster_by_name(cluster_name)) File "pytests/xdcr/xdcrnewbasetests.py", line 2791, in get_cb_cluster_by_name raise XDCRException("Couchbase Cluster with name: %s not exist" % name) XDCRException: Couchbase Cluster with name: C1 not exist ---------------------------------------------------------------------- Ran 1 test in 22.302s FAILED (errors=1) Cluster instance shutdown with force summary so far suite recreatebuckettests.RecreateMembaseBuckets , pass 0 , fail 1 failures so far... recreatebuckettests.RecreateMembaseBuckets.test_default_moxi summary so far suite deletebuckettests.DeleteMembaseBuckets , pass 1 , fail 0 summary so far suite createbuckettests.CreateMembaseBucketsTests , pass 1 , fail 4 failures so far... createbuckettests.CreateMembaseBucketsTests.test_default_moxi createbuckettests.CreateMembaseBucketsTests.test_default_on_non_default_port createbuckettests.CreateMembaseBucketsTests.test_non_default_case_sensitive_different_port createbuckettests.CreateMembaseBucketsTests.test_two_replica summary so far suite setgettests.MembaseBucket , pass 1 , fail 0 summary so far suite expirytests.ExpiryTests , pass 1 , fail 0 summary so far suite memcapable.GetlTests , pass 2 , fail 0 summary so far suite memorysanitytests.MemorySanity , pass 1 , fail 0 summary so far suite drainratetests.DrainRateTests , pass 1 , fail 0 summary so far suite view.viewquerytests.ViewQueryTests , pass 1 , fail 0 summary so far suite view.createdeleteview.CreateDeleteViewTests , pass 1 , fail 0 summary so far suite rebalance.rebalancein.RebalanceInTests , pass 0 , fail 1 failures so far... rebalance.rebalancein.RebalanceInTests.rebalance_in_with_ops summary so far suite rebalance.rebalanceout.RebalanceOutTests , pass 0 , fail 1 failures so far... rebalance.rebalanceout.RebalanceOutTests.rebalance_out_with_ops summary so far suite failover.failovertests.FailoverTests , pass 0 , fail 1 failures so far... failover.failovertests.FailoverTests.test_failover_normal summary so far suite CCCP.CCCP , pass 0 , fail 2 failures so far... CCCP.CCCP.test_get_config_client CCCP.CCCP.test_not_my_vbucket_config summary so far suite flush.bucketflush.BucketFlushTests , pass 0 , fail 1 failures so far... flush.bucketflush.BucketFlushTests.bucketflush_with_data_ops_moxi summary so far suite security.audittest.auditTest , pass 0 , fail 1 failures so far... security.audittest.auditTest.test_bucketEvents summary so far suite tuqquery.tuq_precedence.PrecedenceTests , pass 0 , fail 2 failures so far... tuqquery.tuq_precedence.PrecedenceTests.test_case_and_like tuqquery.tuq_precedence.PrecedenceTests.test_case_and_like summary so far suite tuqquery.tuq_index.QueriesViewsTests , pass 0 , fail 3 failures so far... tuqquery.tuq_index.QueriesViewsTests.test_primary_create_delete_index tuqquery.tuq_index.QueriesViewsTests.test_primary_create_delete_index tuqquery.tuq_index.QueriesViewsTests.test_explain_index_attr summary so far suite tuqquery.tuq_dml.DMLQueryTests , pass 0 , fail 1 failures so far... tuqquery.tuq_dml.DMLQueryTests.test_sanity summary so far suite 2i.indexscans_2i.SecondaryIndexingScanTests , pass 0 , fail 1 failures so far... 2i.indexscans_2i.SecondaryIndexingScanTests.test_multi_create_query_explain_drop_index summary so far suite xdcr.uniXDCR.unidirectional , pass 0 , fail 1 failures so far... xdcr.uniXDCR.unidirectional.load_with_ops summary so far suite xdcr.filterXDCR.XDCRFilterTests , pass 1 , fail 1 failures so far... xdcr.filterXDCR.XDCRFilterTests.test_xdcr_with_filter summary so far suite xdcr.biXDCR.bidirectional , pass 0 , fail 1 failures so far... xdcr.biXDCR.bidirectional.load_with_async_ops testrunner logs, diags and results are available under /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_33 Logs will be stored at /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_34 ./testrunner -i ./4node.ini -t fts.stable_topology_fts.StableTopFTS.run_default_index_query,items=1000,query="{\"match\": \"safiya@mcdiabetes.com\", \"field\":\"email\"}",expected_hits=1000,cluster=D+F,F Test Input params: {'items': '1000', 'expected_hits': '1000', 'conf_file': './4node.conf', 'num_nodes': 4, 'cluster_name': '4node', 'cluster': 'D+F,F', 'ini': './4node.ini', 'case_number': 34, 'query': '"{\\"match\\": \\"safiya@mcdiabetes.com\\", \\"field\\":\\"email\\"}"', 'logs_folder': '/testrunner/logs/testrunner-18-Jan-25_21-16-28/test_34', 'spec': '4node'} [2018-01-25 21:56:10,073] - [fts_base:2828] INFO - ==== FTSbasetests setup is started for test #34 run_default_index_query ==== [2018-01-25 21:56:10,086] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:56:10,086] - [fts_base:1725] INFO - removing nodes from cluster ... [2018-01-25 21:56:10,094] - [fts_base:1727] INFO - cleanup [ip:172.17.0.8 port:8091 ssh_username:root] [2018-01-25 21:56:10,102] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.8 [2018-01-25 21:56:10,103] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:56:12,126] - [cluster_helper:254] INFO - rebalancing all nodes in order to remove nodes [2018-01-25 21:56:12,130] - [rest_client:1423] INFO - rebalance params : {'password': 'password', 'ejectedNodes': u'ns_1@172.17.0.7,ns_1@172.17.0.5,ns_1@172.17.0.6', 'user': 'Administrator', 'knownNodes': u'ns_1@cb-example-0003.cb-example.default.svc,ns_1@172.17.0.7,ns_1@172.17.0.5,ns_1@172.17.0.6'} [2018-01-25 21:56:12,142] - [rest_client:807] ERROR - POST http://172.17.0.8:8091/controller/rebalance body: password=password&ejectedNodes=ns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6&user=Administrator&knownNodes=ns_1%40cb-example-0003.cb-example.default.svc%2Cns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6 headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 400 reason: status: 400, content: No active nodes left No active nodes left auth: Administrator:password [2018-01-25 21:56:12,143] - [rest_client:1430] ERROR - rebalance operation failed: No active nodes left run_default_index_query (fts.stable_topology_fts.StableTopFTS) ... ERROR ====================================================================== ERROR: run_default_index_query (fts.stable_topology_fts.StableTopFTS) ---------------------------------------------------------------------- Traceback (most recent call last): File "pytests/fts/stable_topology_fts.py", line 17, in setUp super(StableTopFTS, self).setUp() File "pytests/fts/fts_base.py", line 2833, in setUp self.__setup_for_test() File "pytests/fts/fts_base.py", line 2933, in __setup_for_test self.__cleanup_previous() File "pytests/fts/fts_base.py", line 3092, in __cleanup_previous self._cb_cluster.cleanup_cluster(self, cluster_shutdown=False) File "pytests/fts/fts_base.py", line 1742, in cleanup_cluster ClusterOperationHelper.cleanup_cluster([node]) File "lib/membase/helper/cluster_helper.py", line 259, in cleanup_cluster wait_for_rebalance=wait_for_rebalance) File "lib/membase/api/rest_client.py", line 110, in remove_nodes self.rest.rebalance(knownNodes, ejectedNodes) File "lib/membase/api/rest_client.py", line 1433, in rebalance parameters=params) InvalidArgumentException: controller/rebalance with error message No active nodes left failed when invoked with parameters: password=password&ejectedNodes=ns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6&user=Administrator&knownNodes=ns_1%40cb-example-0003.cb-example.default.svc%2Cns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6 ---------------------------------------------------------------------- Ran 1 test in 2.072s FAILED (errors=1) summary so far suite recreatebuckettests.RecreateMembaseBuckets , pass 0 , fail 1 failures so far... recreatebuckettests.RecreateMembaseBuckets.test_default_moxi summary so far suite deletebuckettests.DeleteMembaseBuckets , pass 1 , fail 0 summary so far suite createbuckettests.CreateMembaseBucketsTests , pass 1 , fail 4 failures so far... createbuckettests.CreateMembaseBucketsTests.test_default_moxi createbuckettests.CreateMembaseBucketsTests.test_default_on_non_default_port createbuckettests.CreateMembaseBucketsTests.test_non_default_case_sensitive_different_port createbuckettests.CreateMembaseBucketsTests.test_two_replica summary so far suite setgettests.MembaseBucket , pass 1 , fail 0 summary so far suite expirytests.ExpiryTests , pass 1 , fail 0 summary so far suite memcapable.GetlTests , pass 2 , fail 0 summary so far suite memorysanitytests.MemorySanity , pass 1 , fail 0 summary so far suite drainratetests.DrainRateTests , pass 1 , fail 0 summary so far suite view.viewquerytests.ViewQueryTests , pass 1 , fail 0 summary so far suite view.createdeleteview.CreateDeleteViewTests , pass 1 , fail 0 summary so far suite rebalance.rebalancein.RebalanceInTests , pass 0 , fail 1 failures so far... rebalance.rebalancein.RebalanceInTests.rebalance_in_with_ops summary so far suite rebalance.rebalanceout.RebalanceOutTests , pass 0 , fail 1 failures so far... rebalance.rebalanceout.RebalanceOutTests.rebalance_out_with_ops summary so far suite failover.failovertests.FailoverTests , pass 0 , fail 1 failures so far... failover.failovertests.FailoverTests.test_failover_normal summary so far suite CCCP.CCCP , pass 0 , fail 2 failures so far... CCCP.CCCP.test_get_config_client CCCP.CCCP.test_not_my_vbucket_config summary so far suite flush.bucketflush.BucketFlushTests , pass 0 , fail 1 failures so far... flush.bucketflush.BucketFlushTests.bucketflush_with_data_ops_moxi summary so far suite security.audittest.auditTest , pass 0 , fail 1 failures so far... security.audittest.auditTest.test_bucketEvents summary so far suite tuqquery.tuq_precedence.PrecedenceTests , pass 0 , fail 2 failures so far... tuqquery.tuq_precedence.PrecedenceTests.test_case_and_like tuqquery.tuq_precedence.PrecedenceTests.test_case_and_like summary so far suite tuqquery.tuq_index.QueriesViewsTests , pass 0 , fail 3 failures so far... tuqquery.tuq_index.QueriesViewsTests.test_primary_create_delete_index tuqquery.tuq_index.QueriesViewsTests.test_primary_create_delete_index tuqquery.tuq_index.QueriesViewsTests.test_explain_index_attr summary so far suite tuqquery.tuq_dml.DMLQueryTests , pass 0 , fail 1 failures so far... tuqquery.tuq_dml.DMLQueryTests.test_sanity summary so far suite 2i.indexscans_2i.SecondaryIndexingScanTests , pass 0 , fail 1 failures so far... 2i.indexscans_2i.SecondaryIndexingScanTests.test_multi_create_query_explain_drop_index summary so far suite xdcr.uniXDCR.unidirectional , pass 0 , fail 1 failures so far... xdcr.uniXDCR.unidirectional.load_with_ops summary so far suite xdcr.filterXDCR.XDCRFilterTests , pass 1 , fail 1 failures so far... xdcr.filterXDCR.XDCRFilterTests.test_xdcr_with_filter summary so far suite xdcr.biXDCR.bidirectional , pass 0 , fail 1 failures so far... xdcr.biXDCR.bidirectional.load_with_async_ops summary so far suite fts.stable_topology_fts.StableTopFTS , pass 0 , fail 1 failures so far... fts.stable_topology_fts.StableTopFTS.run_default_index_query testrunner logs, diags and results are available under /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_34 Logs will be stored at /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_35 ./testrunner -i ./4node.ini -t ent_backup_restore.enterprise_backup_restore_test.EnterpriseBackupRestoreTest.test_backup_restore_sanity,items=1000 Test Input params: {'cluster_name': '4node', 'ini': './4node.ini', 'case_number': 35, 'items': '1000', 'logs_folder': '/testrunner/logs/testrunner-18-Jan-25_21-16-28/test_35', 'spec': '4node', 'conf_file': './4node.conf', 'num_nodes': 4} [2018-01-25 21:56:12,241] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 21:56:12,372] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 21:56:12,811] - [rest_client:1964] INFO - Node version in cluster 5.0.1-5003-enterprise [2018-01-25 21:56:12,828] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:56:12,828] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:56:12,828] - [basetestcase:201] INFO - ============== basetestcase setup was started for test #35 test_backup_restore_sanity============== [2018-01-25 21:56:12,855] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.8 [2018-01-25 21:56:12,856] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:56:14,871] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.6 [2018-01-25 21:56:14,872] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:56:16,890] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.7 [2018-01-25 21:56:16,890] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:56:18,901] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.5 [2018-01-25 21:56:18,901] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 21:56:20,919] - [cluster_helper:254] INFO - rebalancing all nodes in order to remove nodes [2018-01-25 21:56:20,925] - [rest_client:1423] INFO - rebalance params : {'password': 'password', 'ejectedNodes': u'ns_1@172.17.0.7,ns_1@172.17.0.5,ns_1@172.17.0.6', 'user': 'Administrator', 'knownNodes': u'ns_1@cb-example-0003.cb-example.default.svc,ns_1@172.17.0.7,ns_1@172.17.0.5,ns_1@172.17.0.6'} [2018-01-25 21:56:20,943] - [rest_client:807] ERROR - POST http://172.17.0.8:8091/controller/rebalance body: password=password&ejectedNodes=ns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6&user=Administrator&knownNodes=ns_1%40cb-example-0003.cb-example.default.svc%2Cns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6 headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 400 reason: status: 400, content: No active nodes left No active nodes left auth: Administrator:password [2018-01-25 21:56:20,945] - [rest_client:1430] ERROR - rebalance operation failed: No active nodes left test_backup_restore_sanity (ent_backup_restore.enterprise_backup_restore_test.EnterpriseBackupRestoreTest) ... Traceback (most recent call last): File "pytests/basetestcase.py", line 203, in setUp self._cluster_cleanup() File "pytests/basetestcase.py", line 494, in _cluster_cleanup ClusterOperationHelper.cleanup_cluster(self.servers, master=self.master) File "lib/membase/helper/cluster_helper.py", line 259, in cleanup_cluster wait_for_rebalance=wait_for_rebalance) File "lib/membase/api/rest_client.py", line 110, in remove_nodes self.rest.rebalance(knownNodes, ejectedNodes) File "lib/membase/api/rest_client.py", line 1433, in rebalance parameters=params) InvalidArgumentException: controller/rebalance with error message No active nodes left failed when invoked with parameters: password=password&ejectedNodes=ns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6&user=Administrator&knownNodes=ns_1%40cb-example-0003.cb-example.default.svc%2Cns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6 ERROR ====================================================================== ERROR: test_backup_restore_sanity (ent_backup_restore.enterprise_backup_restore_test.EnterpriseBackupRestoreTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "pytests/ent_backup_restore/enterprise_backup_restore_test.py", line 54, in setUp super(EnterpriseBackupRestoreTest, self).setUp() File "pytests/ent_backup_restore/enterprise_backup_restore_base.py", line 57, in setUp super(EnterpriseBackupRestoreBase, self).setUp() File "pytests/upgrade/newupgradebasetest.py", line 36, in setUp super(NewUpgradeBaseTest, self).setUp() File "pytests/basetestcase.py", line 352, in setUp self.fail(e) AssertionError: controller/rebalance with error message No active nodes left failed when invoked with parameters: password=password&ejectedNodes=ns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6&user=Administrator&knownNodes=ns_1%40cb-example-0003.cb-example.default.svc%2Cns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6 ---------------------------------------------------------------------- Ran 1 test in 8.705s FAILED (errors=1) Cluster instance shutdown with force summary so far suite recreatebuckettests.RecreateMembaseBuckets , pass 0 , fail 1 failures so far... recreatebuckettests.RecreateMembaseBuckets.test_default_moxi summary so far suite deletebuckettests.DeleteMembaseBuckets , pass 1 , fail 0 summary so far suite createbuckettests.CreateMembaseBucketsTests , pass 1 , fail 4 failures so far... createbuckettests.CreateMembaseBucketsTests.test_default_moxi createbuckettests.CreateMembaseBucketsTests.test_default_on_non_default_port createbuckettests.CreateMembaseBucketsTests.test_non_default_case_sensitive_different_port createbuckettests.CreateMembaseBucketsTests.test_two_replica summary so far suite setgettests.MembaseBucket , pass 1 , fail 0 summary so far suite expirytests.ExpiryTests , pass 1 , fail 0 summary so far suite memcapable.GetlTests , pass 2 , fail 0 summary so far suite memorysanitytests.MemorySanity , pass 1 , fail 0 summary so far suite drainratetests.DrainRateTests , pass 1 , fail 0 summary so far suite view.viewquerytests.ViewQueryTests , pass 1 , fail 0 summary so far suite view.createdeleteview.CreateDeleteViewTests , pass 1 , fail 0 summary so far suite rebalance.rebalancein.RebalanceInTests , pass 0 , fail 1 failures so far... rebalance.rebalancein.RebalanceInTests.rebalance_in_with_ops summary so far suite rebalance.rebalanceout.RebalanceOutTests , pass 0 , fail 1 failures so far... rebalance.rebalanceout.RebalanceOutTests.rebalance_out_with_ops summary so far suite failover.failovertests.FailoverTests , pass 0 , fail 1 failures so far... failover.failovertests.FailoverTests.test_failover_normal summary so far suite CCCP.CCCP , pass 0 , fail 2 failures so far... CCCP.CCCP.test_get_config_client CCCP.CCCP.test_not_my_vbucket_config summary so far suite flush.bucketflush.BucketFlushTests , pass 0 , fail 1 failures so far... flush.bucketflush.BucketFlushTests.bucketflush_with_data_ops_moxi summary so far suite security.audittest.auditTest , pass 0 , fail 1 failures so far... security.audittest.auditTest.test_bucketEvents summary so far suite tuqquery.tuq_precedence.PrecedenceTests , pass 0 , fail 2 failures so far... tuqquery.tuq_precedence.PrecedenceTests.test_case_and_like tuqquery.tuq_precedence.PrecedenceTests.test_case_and_like summary so far suite tuqquery.tuq_index.QueriesViewsTests , pass 0 , fail 3 failures so far... tuqquery.tuq_index.QueriesViewsTests.test_primary_create_delete_index tuqquery.tuq_index.QueriesViewsTests.test_primary_create_delete_index tuqquery.tuq_index.QueriesViewsTests.test_explain_index_attr summary so far suite tuqquery.tuq_dml.DMLQueryTests , pass 0 , fail 1 failures so far... tuqquery.tuq_dml.DMLQueryTests.test_sanity summary so far suite 2i.indexscans_2i.SecondaryIndexingScanTests , pass 0 , fail 1 failures so far... 2i.indexscans_2i.SecondaryIndexingScanTests.test_multi_create_query_explain_drop_index summary so far suite xdcr.uniXDCR.unidirectional , pass 0 , fail 1 failures so far... xdcr.uniXDCR.unidirectional.load_with_ops summary so far suite xdcr.filterXDCR.XDCRFilterTests , pass 1 , fail 1 failures so far... xdcr.filterXDCR.XDCRFilterTests.test_xdcr_with_filter summary so far suite xdcr.biXDCR.bidirectional , pass 0 , fail 1 failures so far... xdcr.biXDCR.bidirectional.load_with_async_ops summary so far suite fts.stable_topology_fts.StableTopFTS , pass 0 , fail 1 failures so far... fts.stable_topology_fts.StableTopFTS.run_default_index_query summary so far suite ent_backup_restore.enterprise_backup_restore_test.EnterpriseBackupRestoreTest , pass 0 , fail 1 failures so far... ent_backup_restore.enterprise_backup_restore_test.EnterpriseBackupRestoreTest.test_backup_restore_sanity testrunner logs, diags and results are available under /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_35 Logs will be stored at /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_36 ./testrunner -i ./4node.ini -t tuqquery.tuq_2i_index.QueriesIndexTests.test_covering_index,covering_index=true,doc-per-day=1,skip_index=True,index_type=gsi Test Input params: {'covering_index': 'true', 'doc-per-day': '1', 'conf_file': './4node.conf', 'num_nodes': 4, 'cluster_name': '4node', 'index_type': 'gsi', 'ini': './4node.ini', 'skip_index': 'True', 'case_number': 36, 'logs_folder': '/testrunner/logs/testrunner-18-Jan-25_21-16-28/test_36', 'spec': '4node'} [2018-01-25 21:56:21,069] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 21:56:21,208] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 21:56:21,757] - [rest_client:1964] INFO - Node version in cluster 5.0.1-5003-enterprise [2018-01-25 21:56:21,767] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 21:56:21,767] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 21:56:21,767] - [basetestcase:201] INFO - ============== basetestcase setup was started for test #36 test_covering_index============== [2018-01-25 21:56:21,768] - [basetestcase:230] INFO - any cluster operation in setup will be skipped [2018-01-25 21:56:21,768] - [basetestcase:233] INFO - ============== basetestcase setup was finished for test #36 test_covering_index ============== [2018-01-25 21:56:21,768] - [tuq:41] INFO - ============== QueryTests setup has started ============== [2018-01-25 21:56:21,768] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 21:56:21,899] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 21:56:22,393] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 21:56:22,474] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 21:56:23,186] - [tuq:563] INFO - Generating default:default data... [2018-01-25 21:56:24,587] - [tuq:114] INFO - ---------------------------------------------------------------------------------------------------- [2018-01-25 21:56:24,587] - [tuq:115] INFO - Temp fix for MB-16888 [2018-01-25 21:56:24,587] - [remote_util:2995] INFO - running command.raw on 172.17.0.8: killall -9 cbq-engine [2018-01-25 21:56:24,593] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:56:24,593] - [remote_util:2995] INFO - running command.raw on 172.17.0.8: killall -9 indexer [2018-01-25 21:56:24,664] - [remote_util:3032] INFO - command executed successfully [2018-01-25 21:56:24,665] - [basetestcase:482] INFO - sleep for 30 secs. wait for indexer ... [2018-01-25 21:56:54,910] - [tuq:120] INFO - ---------------------------------------------------------------------------------------------------- [2018-01-25 21:56:54,959] - [tuq:127] INFO - ============== QueryTests setup has completed ============== [2018-01-25 21:56:54,960] - [tuq_2i_index:18] INFO - ============== QueriesIndexTests setup has started ============== [2018-01-25 21:56:55,005] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 21:56:55,176] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 21:56:55,581] - [tuq_2i_index:27] INFO - ============== QueriesIndexTests setup has completed ============== [2018-01-25 21:56:55,581] - [tuq:185] INFO - ============== System Config: ============== [2018-01-25 21:56:55,587] - [tuq:756] INFO - RUN QUERY SELECT * FROM system:indexes [2018-01-25 21:56:55,587] - [rest_client:3183] INFO - query params : scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes [2018-01-25 21:56:55,587] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:56:58,592] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:57:01,596] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:57:04,600] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:57:07,606] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:57:10,610] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:57:13,613] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:57:16,616] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:57:19,621] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:57:22,623] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:57:25,627] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:57:28,628] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:57:31,632] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:57:34,635] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:57:37,636] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:57:40,640] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:57:43,644] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:57:46,649] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:57:49,651] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:57:52,832] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:57:55,837] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:57:58,838] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:58:01,843] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:58:04,844] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:58:07,849] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:58:10,853] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:58:13,858] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:58:16,861] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:58:19,865] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:58:22,869] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:58:25,871] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:58:28,875] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:58:31,877] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:58:34,881] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:58:37,886] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:58:40,890] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:58:43,895] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:58:46,900] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:58:49,901] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:58:52,905] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:58:55,912] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:58:58,917] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:59:01,923] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:59:04,929] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:59:07,935] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:59:10,941] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:59:13,945] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:59:16,949] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:59:19,951] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:59:22,953] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:59:25,957] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:59:28,966] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:59:31,971] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:59:34,975] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:59:40,196] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:59:43,203] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:59:46,207] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:59:49,211] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:59:52,213] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:59:55,214] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 21:59:58,219] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:00:01,222] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:00:04,225] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:00:07,931] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:00:10,945] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:00:13,950] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:00:17,815] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:00:20,840] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:00:23,845] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:00:26,849] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:00:29,853] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:00:32,857] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:00:35,860] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:00:38,864] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:00:41,866] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:00:44,870] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:00:47,874] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:00:50,878] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:00:53,882] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:00:56,884] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:00:59,887] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:01:02,890] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:01:05,893] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:01:08,898] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:01:11,901] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:01:14,905] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:01:17,909] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:01:20,914] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:01:23,919] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:01:26,921] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:01:29,925] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:01:32,930] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:01:35,933] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:01:38,936] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:01:41,940] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:01:44,944] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:01:47,946] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:01:50,950] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:01:53,951] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:01:56,955] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:01:59,959] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:02:02,964] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:02:05,968] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:02:08,972] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:02:11,977] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:02:14,978] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:02:17,979] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:02:20,983] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:02:23,987] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:02:26,992] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:02:29,996] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:02:33,001] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:02:36,005] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:02:39,008] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:02:42,013] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:02:45,014] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:02:48,017] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:02:51,018] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:02:54,021] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:02:57,025] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:03:00,029] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:03:03,033] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:03:06,040] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:03:09,043] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:03:12,047] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:03:15,048] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:03:18,052] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:03:21,056] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:03:24,057] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:03:27,061] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:03:30,065] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:03:33,068] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:03:36,072] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:03:39,073] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:03:42,074] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:03:45,079] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:03:48,084] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:03:51,087] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:03:54,090] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:03:57,094] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:04:00,098] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:04:03,101] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:04:06,105] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:04:09,108] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:04:12,112] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:04:15,116] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:04:18,120] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:04:21,122] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:04:24,126] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:04:27,130] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:04:30,135] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:04:33,139] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:04:36,144] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:04:39,148] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:04:42,152] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:04:45,154] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:04:48,158] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:04:51,161] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:04:54,166] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:04:57,169] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:05:00,171] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:05:03,173] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:05:06,177] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:05:09,181] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:05:12,186] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:05:15,192] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:05:18,195] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:05:21,199] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:05:24,202] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:05:27,206] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:05:30,208] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:05:33,212] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:05:36,217] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:05:39,221] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:05:42,224] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:05:45,228] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:05:48,232] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:05:51,236] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:05:54,240] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:05:57,244] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:06:00,248] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:06:03,252] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:06:06,254] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:06:09,258] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:06:12,262] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:06:15,266] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:06:18,269] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:06:21,271] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:06:24,275] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:06:27,280] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:06:30,284] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:06:33,289] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:06:36,293] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:06:39,297] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:06:42,301] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:06:45,304] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:06:48,306] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:06:51,311] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:06:54,313] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:06:57,319] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:07:00,322] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:07:03,326] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:07:06,328] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:07:09,330] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:07:12,334] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:07:15,338] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:07:18,342] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:07:21,345] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:07:24,350] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:07:27,351] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:07:30,354] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:07:33,358] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:07:36,359] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:07:39,363] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:07:42,365] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:07:45,369] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:07:48,373] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:07:51,377] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:07:54,381] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:07:57,385] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:08:00,389] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:08:03,393] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:08:06,398] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:08:09,401] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:08:12,404] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:08:15,408] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:08:18,409] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:08:21,414] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:08:24,418] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:08:27,422] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:08:30,424] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:08:33,425] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:08:36,429] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:08:39,432] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:08:42,435] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:08:45,439] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:08:48,443] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:08:51,447] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:08:54,452] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:08:57,454] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:09:00,455] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:09:03,460] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:09:06,464] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:09:09,468] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:09:12,472] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:09:15,477] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:09:18,481] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:09:21,485] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:09:24,486] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:09:27,490] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:09:30,495] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:09:33,499] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:09:36,503] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:09:39,505] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:09:42,506] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:09:45,510] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:09:48,511] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:09:51,515] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:09:54,517] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:09:57,520] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:10:00,524] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:10:03,526] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:10:06,526] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:10:09,528] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:10:12,532] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:10:15,534] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:10:18,537] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:10:21,538] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:10:24,543] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:10:27,546] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:10:30,549] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:10:33,550] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:10:36,555] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:10:39,560] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:10:42,563] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:10:45,567] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:10:48,570] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:10:51,574] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:10:54,578] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:10:57,581] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:11:00,585] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:11:03,590] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:11:06,593] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:11:09,598] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:11:12,602] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:11:15,605] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:11:18,609] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:11:21,611] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:11:24,615] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:11:27,619] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:11:30,623] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:11:33,624] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:11:36,628] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:11:39,631] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:11:42,636] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:11:45,640] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:11:48,645] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:11:51,651] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:11:54,652] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:11:57,657] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:12:00,660] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:12:03,664] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:12:06,666] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:12:09,670] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:12:12,674] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:12:15,683] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:12:18,689] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:12:21,694] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:12:24,699] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:12:27,703] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:12:30,707] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:12:33,711] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:12:36,714] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:12:39,718] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:12:42,721] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:12:45,727] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:12:48,728] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:12:51,731] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:12:54,735] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:12:57,738] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:13:00,742] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:13:03,747] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:13:06,750] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:13:09,752] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:13:12,755] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:13:15,759] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:13:18,763] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:13:21,767] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:13:24,771] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:13:27,775] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:13:30,779] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:13:33,783] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:13:36,786] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:13:39,790] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:13:42,795] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:13:45,800] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:13:48,804] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:13:51,808] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:13:54,809] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:13:57,813] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:14:00,817] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:14:03,821] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:14:06,825] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:14:09,827] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:14:12,831] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:14:15,833] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:14:18,836] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:14:21,840] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:14:24,843] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:14:27,845] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:14:30,850] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:14:33,851] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:14:36,853] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:14:39,857] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:14:42,862] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:14:45,864] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:14:48,868] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:14:51,872] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:14:54,877] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:14:57,881] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:15:00,885] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:15:03,907] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:15:06,912] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:15:09,918] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:15:12,923] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:15:15,927] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:15:18,931] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:15:21,935] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:15:24,939] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:15:27,942] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:15:30,946] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:15:33,950] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:15:36,954] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:15:39,957] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:15:42,961] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:15:45,965] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:15:48,970] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:15:51,974] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:15:54,978] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:15:57,983] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:16:00,986] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:16:03,990] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:16:06,993] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:16:09,997] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:16:12,998] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:16:16,001] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:16:19,005] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:16:22,009] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:16:25,013] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:16:28,017] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:16:31,021] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:16:34,025] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:16:37,028] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:16:40,032] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:16:43,033] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:16:46,034] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:16:49,038] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:16:52,040] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:16:55,044] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:16:58,046] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:17:01,050] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:17:04,054] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:17:07,059] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:17:10,061] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:17:13,065] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:17:16,069] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:17:19,070] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:17:22,074] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:17:25,079] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:17:28,084] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:17:31,089] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:17:34,093] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:17:37,097] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:17:40,101] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:17:43,105] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:17:46,108] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:17:49,111] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:17:52,116] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:17:55,120] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:17:58,124] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:18:01,128] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:18:04,132] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:18:07,133] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:18:10,137] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:18:13,141] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:18:16,144] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:18:19,149] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:18:22,152] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:18:25,157] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:18:28,159] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:18:31,162] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:18:34,166] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:18:37,170] - [rest_client:811] ERROR - socket error while connecting to http://172.17.0.8:8093/query?scan_consistency=REQUEST_PLUS&statement=SELECT+%2A+FROM+system%3Aindexes error [Errno 111] Connection refused [2018-01-25 22:18:37,170] - [tuq:210] INFO - ============================================= test_covering_index (tuqquery.tuq_2i_index.QueriesIndexTests) ... FAIL [2018-01-25 22:18:37,172] - [tuq_2i_index:36] INFO - ============== QueriesIndexTests tearDown has started ============== [2018-01-25 22:18:37,172] - [tuq_2i_index:37] INFO - ============== QueriesIndexTests tearDown has completed ============== [2018-01-25 22:18:37,173] - [tuq:153] INFO - ============== QueryTests tearDown has started ============== [2018-01-25 22:18:37,173] - [tuq:169] INFO - ============== QueryTests tearDown has completed ============== ====================================================================== FAIL: test_covering_index (tuqquery.tuq_2i_index.QueriesIndexTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "pytests/tuqquery/tuq_2i_index.py", line 4383, in test_covering_index self.fail_if_no_buckets() File "pytests/tuqquery/tuq.py", line 217, in fail_if_no_buckets self.fail('FAIL: This test requires buckets') AssertionError: FAIL: This test requires buckets ---------------------------------------------------------------------- Ran 1 test in 1336.113s FAILED (failures=1) Cluster instance shutdown with force summary so far suite recreatebuckettests.RecreateMembaseBuckets , pass 0 , fail 1 failures so far... recreatebuckettests.RecreateMembaseBuckets.test_default_moxi summary so far suite deletebuckettests.DeleteMembaseBuckets , pass 1 , fail 0 summary so far suite createbuckettests.CreateMembaseBucketsTests , pass 1 , fail 4 failures so far... createbuckettests.CreateMembaseBucketsTests.test_default_moxi createbuckettests.CreateMembaseBucketsTests.test_default_on_non_default_port createbuckettests.CreateMembaseBucketsTests.test_non_default_case_sensitive_different_port createbuckettests.CreateMembaseBucketsTests.test_two_replica summary so far suite setgettests.MembaseBucket , pass 1 , fail 0 summary so far suite expirytests.ExpiryTests , pass 1 , fail 0 summary so far suite memcapable.GetlTests , pass 2 , fail 0 summary so far suite memorysanitytests.MemorySanity , pass 1 , fail 0 summary so far suite drainratetests.DrainRateTests , pass 1 , fail 0 summary so far suite view.viewquerytests.ViewQueryTests , pass 1 , fail 0 summary so far suite view.createdeleteview.CreateDeleteViewTests , pass 1 , fail 0 summary so far suite rebalance.rebalancein.RebalanceInTests , pass 0 , fail 1 failures so far... rebalance.rebalancein.RebalanceInTests.rebalance_in_with_ops summary so far suite rebalance.rebalanceout.RebalanceOutTests , pass 0 , fail 1 failures so far... rebalance.rebalanceout.RebalanceOutTests.rebalance_out_with_ops summary so far suite failover.failovertests.FailoverTests , pass 0 , fail 1 failures so far... failover.failovertests.FailoverTests.test_failover_normal summary so far suite CCCP.CCCP , pass 0 , fail 2 failures so far... CCCP.CCCP.test_get_config_client CCCP.CCCP.test_not_my_vbucket_config summary so far suite flush.bucketflush.BucketFlushTests , pass 0 , fail 1 failures so far... flush.bucketflush.BucketFlushTests.bucketflush_with_data_ops_moxi summary so far suite security.audittest.auditTest , pass 0 , fail 1 failures so far... security.audittest.auditTest.test_bucketEvents summary so far suite tuqquery.tuq_precedence.PrecedenceTests , pass 0 , fail 2 failures so far... tuqquery.tuq_precedence.PrecedenceTests.test_case_and_like tuqquery.tuq_precedence.PrecedenceTests.test_case_and_like summary so far suite tuqquery.tuq_index.QueriesViewsTests , pass 0 , fail 3 failures so far... tuqquery.tuq_index.QueriesViewsTests.test_primary_create_delete_index tuqquery.tuq_index.QueriesViewsTests.test_primary_create_delete_index tuqquery.tuq_index.QueriesViewsTests.test_explain_index_attr summary so far suite tuqquery.tuq_dml.DMLQueryTests , pass 0 , fail 1 failures so far... tuqquery.tuq_dml.DMLQueryTests.test_sanity summary so far suite 2i.indexscans_2i.SecondaryIndexingScanTests , pass 0 , fail 1 failures so far... 2i.indexscans_2i.SecondaryIndexingScanTests.test_multi_create_query_explain_drop_index summary so far suite xdcr.uniXDCR.unidirectional , pass 0 , fail 1 failures so far... xdcr.uniXDCR.unidirectional.load_with_ops summary so far suite xdcr.filterXDCR.XDCRFilterTests , pass 1 , fail 1 failures so far... xdcr.filterXDCR.XDCRFilterTests.test_xdcr_with_filter summary so far suite xdcr.biXDCR.bidirectional , pass 0 , fail 1 failures so far... xdcr.biXDCR.bidirectional.load_with_async_ops summary so far suite fts.stable_topology_fts.StableTopFTS , pass 0 , fail 1 failures so far... fts.stable_topology_fts.StableTopFTS.run_default_index_query summary so far suite ent_backup_restore.enterprise_backup_restore_test.EnterpriseBackupRestoreTest , pass 0 , fail 1 failures so far... ent_backup_restore.enterprise_backup_restore_test.EnterpriseBackupRestoreTest.test_backup_restore_sanity summary so far suite tuqquery.tuq_2i_index.QueriesIndexTests , pass 0 , fail 1 failures so far... tuqquery.tuq_2i_index.QueriesIndexTests.test_covering_index testrunner logs, diags and results are available under /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_36 Logs will be stored at /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_37 ./testrunner -i ./4node.ini -t subdoc.subdoc_nested_dataset.SubdocNestedDataset.test_sanity Test Input params: {'conf_file': './4node.conf', 'num_nodes': 4, 'cluster_name': '4node', 'ini': './4node.ini', 'case_number': 37, 'logs_folder': '/testrunner/logs/testrunner-18-Jan-25_21-16-28/test_37', 'spec': '4node'} [2018-01-25 22:18:37,221] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 22:18:37,356] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 22:18:37,866] - [rest_client:1964] INFO - Node version in cluster 5.0.1-5003-enterprise [2018-01-25 22:18:37,875] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 22:18:37,875] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 22:18:37,875] - [basetestcase:201] INFO - ============== basetestcase setup was started for test #37 test_sanity============== [2018-01-25 22:18:37,894] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.8 [2018-01-25 22:18:37,895] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 22:18:39,904] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.6 [2018-01-25 22:18:39,904] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 22:18:41,912] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.7 [2018-01-25 22:18:41,912] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 22:18:43,923] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.5 [2018-01-25 22:18:43,924] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 22:18:45,942] - [cluster_helper:254] INFO - rebalancing all nodes in order to remove nodes [2018-01-25 22:18:45,944] - [rest_client:1423] INFO - rebalance params : {'password': 'password', 'ejectedNodes': u'ns_1@172.17.0.7,ns_1@172.17.0.5,ns_1@172.17.0.6', 'user': 'Administrator', 'knownNodes': u'ns_1@cb-example-0003.cb-example.default.svc,ns_1@172.17.0.7,ns_1@172.17.0.5,ns_1@172.17.0.6'} [2018-01-25 22:18:45,947] - [rest_client:807] ERROR - POST http://172.17.0.8:8091/controller/rebalance body: password=password&ejectedNodes=ns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6&user=Administrator&knownNodes=ns_1%40cb-example-0003.cb-example.default.svc%2Cns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6 headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 400 reason: status: 400, content: No active nodes left No active nodes left auth: Administrator:password [2018-01-25 22:18:45,947] - [rest_client:1430] ERROR - rebalance operation failed: No active nodes left test_sanity (subdoc.subdoc_nested_dataset.SubdocNestedDataset) ... Traceback (most recent call last): File "pytests/basetestcase.py", line 203, in setUp self._cluster_cleanup() File "pytests/basetestcase.py", line 494, in _cluster_cleanup ClusterOperationHelper.cleanup_cluster(self.servers, master=self.master) File "lib/membase/helper/cluster_helper.py", line 259, in cleanup_cluster wait_for_rebalance=wait_for_rebalance) File "lib/membase/api/rest_client.py", line 110, in remove_nodes self.rest.rebalance(knownNodes, ejectedNodes) File "lib/membase/api/rest_client.py", line 1433, in rebalance parameters=params) InvalidArgumentException: controller/rebalance with error message No active nodes left failed when invoked with parameters: password=password&ejectedNodes=ns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6&user=Administrator&knownNodes=ns_1%40cb-example-0003.cb-example.default.svc%2Cns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6 ERROR ====================================================================== ERROR: test_sanity (subdoc.subdoc_nested_dataset.SubdocNestedDataset) ---------------------------------------------------------------------- Traceback (most recent call last): File "pytests/subdoc/subdoc_nested_dataset.py", line 12, in setUp super(SubdocNestedDataset, self).setUp() File "pytests/subdoc/subdoc_base.py", line 9, in setUp super(SubdocBaseTest, self).setUp() File "pytests/basetestcase.py", line 352, in setUp self.fail(e) AssertionError: controller/rebalance with error message No active nodes left failed when invoked with parameters: password=password&ejectedNodes=ns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6&user=Administrator&knownNodes=ns_1%40cb-example-0003.cb-example.default.svc%2Cns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6 ---------------------------------------------------------------------- Ran 1 test in 8.727s FAILED (errors=1) Cluster instance shutdown with force summary so far suite recreatebuckettests.RecreateMembaseBuckets , pass 0 , fail 1 failures so far... recreatebuckettests.RecreateMembaseBuckets.test_default_moxi summary so far suite deletebuckettests.DeleteMembaseBuckets , pass 1 , fail 0 summary so far suite createbuckettests.CreateMembaseBucketsTests , pass 1 , fail 4 failures so far... createbuckettests.CreateMembaseBucketsTests.test_default_moxi createbuckettests.CreateMembaseBucketsTests.test_default_on_non_default_port createbuckettests.CreateMembaseBucketsTests.test_non_default_case_sensitive_different_port createbuckettests.CreateMembaseBucketsTests.test_two_replica summary so far suite setgettests.MembaseBucket , pass 1 , fail 0 summary so far suite expirytests.ExpiryTests , pass 1 , fail 0 summary so far suite memcapable.GetlTests , pass 2 , fail 0 summary so far suite memorysanitytests.MemorySanity , pass 1 , fail 0 summary so far suite drainratetests.DrainRateTests , pass 1 , fail 0 summary so far suite view.viewquerytests.ViewQueryTests , pass 1 , fail 0 summary so far suite view.createdeleteview.CreateDeleteViewTests , pass 1 , fail 0 summary so far suite rebalance.rebalancein.RebalanceInTests , pass 0 , fail 1 failures so far... rebalance.rebalancein.RebalanceInTests.rebalance_in_with_ops summary so far suite rebalance.rebalanceout.RebalanceOutTests , pass 0 , fail 1 failures so far... rebalance.rebalanceout.RebalanceOutTests.rebalance_out_with_ops summary so far suite failover.failovertests.FailoverTests , pass 0 , fail 1 failures so far... failover.failovertests.FailoverTests.test_failover_normal summary so far suite CCCP.CCCP , pass 0 , fail 2 failures so far... CCCP.CCCP.test_get_config_client CCCP.CCCP.test_not_my_vbucket_config summary so far suite flush.bucketflush.BucketFlushTests , pass 0 , fail 1 failures so far... flush.bucketflush.BucketFlushTests.bucketflush_with_data_ops_moxi summary so far suite security.audittest.auditTest , pass 0 , fail 1 failures so far... security.audittest.auditTest.test_bucketEvents summary so far suite tuqquery.tuq_precedence.PrecedenceTests , pass 0 , fail 2 failures so far... tuqquery.tuq_precedence.PrecedenceTests.test_case_and_like tuqquery.tuq_precedence.PrecedenceTests.test_case_and_like summary so far suite tuqquery.tuq_index.QueriesViewsTests , pass 0 , fail 3 failures so far... tuqquery.tuq_index.QueriesViewsTests.test_primary_create_delete_index tuqquery.tuq_index.QueriesViewsTests.test_primary_create_delete_index tuqquery.tuq_index.QueriesViewsTests.test_explain_index_attr summary so far suite tuqquery.tuq_dml.DMLQueryTests , pass 0 , fail 1 failures so far... tuqquery.tuq_dml.DMLQueryTests.test_sanity summary so far suite 2i.indexscans_2i.SecondaryIndexingScanTests , pass 0 , fail 1 failures so far... 2i.indexscans_2i.SecondaryIndexingScanTests.test_multi_create_query_explain_drop_index summary so far suite xdcr.uniXDCR.unidirectional , pass 0 , fail 1 failures so far... xdcr.uniXDCR.unidirectional.load_with_ops summary so far suite xdcr.filterXDCR.XDCRFilterTests , pass 1 , fail 1 failures so far... xdcr.filterXDCR.XDCRFilterTests.test_xdcr_with_filter summary so far suite xdcr.biXDCR.bidirectional , pass 0 , fail 1 failures so far... xdcr.biXDCR.bidirectional.load_with_async_ops summary so far suite fts.stable_topology_fts.StableTopFTS , pass 0 , fail 1 failures so far... fts.stable_topology_fts.StableTopFTS.run_default_index_query summary so far suite ent_backup_restore.enterprise_backup_restore_test.EnterpriseBackupRestoreTest , pass 0 , fail 1 failures so far... ent_backup_restore.enterprise_backup_restore_test.EnterpriseBackupRestoreTest.test_backup_restore_sanity summary so far suite tuqquery.tuq_2i_index.QueriesIndexTests , pass 0 , fail 1 failures so far... tuqquery.tuq_2i_index.QueriesIndexTests.test_covering_index summary so far suite subdoc.subdoc_nested_dataset.SubdocNestedDataset , pass 0 , fail 1 failures so far... subdoc.subdoc_nested_dataset.SubdocNestedDataset.test_sanity testrunner logs, diags and results are available under /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_37 Logs will be stored at /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_38 ./testrunner -i ./4node.ini -t tuqquery.tuq_advancedcbqshell.AdvancedQueryTests.test_engine_postive Test Input params: {'conf_file': './4node.conf', 'num_nodes': 4, 'cluster_name': '4node', 'ini': './4node.ini', 'case_number': 38, 'logs_folder': '/testrunner/logs/testrunner-18-Jan-25_21-16-28/test_38', 'spec': '4node'} [2018-01-25 22:18:45,961] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 22:18:46,096] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 22:18:46,542] - [rest_client:1964] INFO - Node version in cluster 5.0.1-5003-enterprise [2018-01-25 22:18:46,552] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 22:18:46,552] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 22:18:46,552] - [basetestcase:201] INFO - ============== basetestcase setup was started for test #38 test_engine_postive============== [2018-01-25 22:18:46,553] - [basetestcase:230] INFO - any cluster operation in setup will be skipped [2018-01-25 22:18:46,553] - [basetestcase:233] INFO - ============== basetestcase setup was finished for test #38 test_engine_postive ============== [2018-01-25 22:18:46,553] - [tuq:41] INFO - ============== QueryTests setup has started ============== [2018-01-25 22:18:46,553] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 22:18:46,683] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 22:18:47,046] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 22:18:47,178] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 22:18:47,651] - [tuq:563] INFO - Generating default:default data... [2018-01-25 22:19:03,632] - [tuq:114] INFO - ---------------------------------------------------------------------------------------------------- [2018-01-25 22:19:03,632] - [tuq:115] INFO - Temp fix for MB-16888 [2018-01-25 22:19:03,632] - [remote_util:2995] INFO - running command.raw on 172.17.0.8: killall -9 cbq-engine [2018-01-25 22:19:03,637] - [remote_util:3032] INFO - command executed successfully [2018-01-25 22:19:03,637] - [remote_util:2995] INFO - running command.raw on 172.17.0.8: killall -9 indexer [2018-01-25 22:19:03,683] - [remote_util:3032] INFO - command executed successfully [2018-01-25 22:19:03,683] - [basetestcase:482] INFO - sleep for 30 secs. wait for indexer ... [2018-01-25 22:19:33,714] - [tuq:120] INFO - ---------------------------------------------------------------------------------------------------- [2018-01-25 22:19:33,715] - [tuq:127] INFO - ============== QueryTests setup has completed ============== [2018-01-25 22:19:33,716] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 22:19:33,851] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 22:19:34,282] - [remote_util:232] INFO - connecting to 172.17.0.6 with username:root password:couchbase ssh_key: [2018-01-25 22:19:34,415] - [remote_util:266] INFO - Connected to 172.17.0.6 [2018-01-25 22:19:34,772] - [remote_util:232] INFO - connecting to 172.17.0.7 with username:root password:couchbase ssh_key: [2018-01-25 22:19:34,857] - [remote_util:266] INFO - Connected to 172.17.0.7 [2018-01-25 22:19:35,330] - [remote_util:232] INFO - connecting to 172.17.0.5 with username:root password:couchbase ssh_key: [2018-01-25 22:19:35,464] - [remote_util:266] INFO - Connected to 172.17.0.5 [2018-01-25 22:19:35,817] - [tuq:153] INFO - ============== QueryTests tearDown has started ============== [2018-01-25 22:19:35,817] - [tuq:169] INFO - ============== QueryTests tearDown has completed ============== test_engine_postive (tuqquery.tuq_advancedcbqshell.AdvancedQueryTests) ... ok ---------------------------------------------------------------------- Ran 1 test in 49.865s OK Cluster instance shutdown with force summary so far suite recreatebuckettests.RecreateMembaseBuckets , pass 0 , fail 1 failures so far... recreatebuckettests.RecreateMembaseBuckets.test_default_moxi summary so far suite deletebuckettests.DeleteMembaseBuckets , pass 1 , fail 0 summary so far suite createbuckettests.CreateMembaseBucketsTests , pass 1 , fail 4 failures so far... createbuckettests.CreateMembaseBucketsTests.test_default_moxi createbuckettests.CreateMembaseBucketsTests.test_default_on_non_default_port createbuckettests.CreateMembaseBucketsTests.test_non_default_case_sensitive_different_port createbuckettests.CreateMembaseBucketsTests.test_two_replica summary so far suite setgettests.MembaseBucket , pass 1 , fail 0 summary so far suite expirytests.ExpiryTests , pass 1 , fail 0 summary so far suite memcapable.GetlTests , pass 2 , fail 0 summary so far suite memorysanitytests.MemorySanity , pass 1 , fail 0 summary so far suite drainratetests.DrainRateTests , pass 1 , fail 0 summary so far suite view.viewquerytests.ViewQueryTests , pass 1 , fail 0 summary so far suite view.createdeleteview.CreateDeleteViewTests , pass 1 , fail 0 summary so far suite rebalance.rebalancein.RebalanceInTests , pass 0 , fail 1 failures so far... rebalance.rebalancein.RebalanceInTests.rebalance_in_with_ops summary so far suite rebalance.rebalanceout.RebalanceOutTests , pass 0 , fail 1 failures so far... rebalance.rebalanceout.RebalanceOutTests.rebalance_out_with_ops summary so far suite failover.failovertests.FailoverTests , pass 0 , fail 1 failures so far... failover.failovertests.FailoverTests.test_failover_normal summary so far suite CCCP.CCCP , pass 0 , fail 2 failures so far... CCCP.CCCP.test_get_config_client CCCP.CCCP.test_not_my_vbucket_config summary so far suite flush.bucketflush.BucketFlushTests , pass 0 , fail 1 failures so far... flush.bucketflush.BucketFlushTests.bucketflush_with_data_ops_moxi summary so far suite security.audittest.auditTest , pass 0 , fail 1 failures so far... security.audittest.auditTest.test_bucketEvents summary so far suite tuqquery.tuq_precedence.PrecedenceTests , pass 0 , fail 2 failures so far... tuqquery.tuq_precedence.PrecedenceTests.test_case_and_like tuqquery.tuq_precedence.PrecedenceTests.test_case_and_like summary so far suite tuqquery.tuq_index.QueriesViewsTests , pass 0 , fail 3 failures so far... tuqquery.tuq_index.QueriesViewsTests.test_primary_create_delete_index tuqquery.tuq_index.QueriesViewsTests.test_primary_create_delete_index tuqquery.tuq_index.QueriesViewsTests.test_explain_index_attr summary so far suite tuqquery.tuq_dml.DMLQueryTests , pass 0 , fail 1 failures so far... tuqquery.tuq_dml.DMLQueryTests.test_sanity summary so far suite 2i.indexscans_2i.SecondaryIndexingScanTests , pass 0 , fail 1 failures so far... 2i.indexscans_2i.SecondaryIndexingScanTests.test_multi_create_query_explain_drop_index summary so far suite xdcr.uniXDCR.unidirectional , pass 0 , fail 1 failures so far... xdcr.uniXDCR.unidirectional.load_with_ops summary so far suite xdcr.filterXDCR.XDCRFilterTests , pass 1 , fail 1 failures so far... xdcr.filterXDCR.XDCRFilterTests.test_xdcr_with_filter summary so far suite xdcr.biXDCR.bidirectional , pass 0 , fail 1 failures so far... xdcr.biXDCR.bidirectional.load_with_async_ops summary so far suite fts.stable_topology_fts.StableTopFTS , pass 0 , fail 1 failures so far... fts.stable_topology_fts.StableTopFTS.run_default_index_query summary so far suite ent_backup_restore.enterprise_backup_restore_test.EnterpriseBackupRestoreTest , pass 0 , fail 1 failures so far... ent_backup_restore.enterprise_backup_restore_test.EnterpriseBackupRestoreTest.test_backup_restore_sanity summary so far suite tuqquery.tuq_2i_index.QueriesIndexTests , pass 0 , fail 1 failures so far... tuqquery.tuq_2i_index.QueriesIndexTests.test_covering_index summary so far suite subdoc.subdoc_nested_dataset.SubdocNestedDataset , pass 0 , fail 1 failures so far... subdoc.subdoc_nested_dataset.SubdocNestedDataset.test_sanity summary so far suite tuqquery.tuq_advancedcbqshell.AdvancedQueryTests , pass 1 , fail 0 testrunner logs, diags and results are available under /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_38 Logs will be stored at /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_39 ./testrunner -i ./4node.ini -t security.x509tests.x509tests.test_basic_ssl_test,default_bucket=False,SSLtype=openssl Test Input params: {'cluster_name': '4node', 'logs_folder': '/testrunner/logs/testrunner-18-Jan-25_21-16-28/test_39', 'SSLtype': 'openssl', 'case_number': 39, 'ini': './4node.ini', 'conf_file': './4node.conf', 'spec': '4node', 'default_bucket': 'False', 'num_nodes': 4} [2018-01-25 22:19:36,287] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 22:19:36,418] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 22:19:36,851] - [rest_client:1964] INFO - Node version in cluster 5.0.1-5003-enterprise [2018-01-25 22:19:36,863] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 22:19:36,864] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 22:19:36,864] - [basetestcase:201] INFO - ============== basetestcase setup was started for test #39 test_basic_ssl_test============== [2018-01-25 22:19:36,876] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.8 [2018-01-25 22:19:36,876] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 22:19:38,886] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.6 [2018-01-25 22:19:38,887] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 22:19:40,898] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.7 [2018-01-25 22:19:40,899] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 22:19:42,911] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.5 [2018-01-25 22:19:42,911] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 22:19:44,938] - [cluster_helper:254] INFO - rebalancing all nodes in order to remove nodes [2018-01-25 22:19:44,941] - [rest_client:1423] INFO - rebalance params : {'password': 'password', 'ejectedNodes': u'ns_1@172.17.0.7,ns_1@172.17.0.5,ns_1@172.17.0.6', 'user': 'Administrator', 'knownNodes': u'ns_1@cb-example-0003.cb-example.default.svc,ns_1@172.17.0.7,ns_1@172.17.0.5,ns_1@172.17.0.6'} [2018-01-25 22:19:44,944] - [rest_client:807] ERROR - POST http://172.17.0.8:8091/controller/rebalance body: password=password&ejectedNodes=ns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6&user=Administrator&knownNodes=ns_1%40cb-example-0003.cb-example.default.svc%2Cns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6 headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 400 reason: status: 400, content: No active nodes left No active nodes left auth: Administrator:password [2018-01-25 22:19:44,945] - [rest_client:1430] ERROR - rebalance operation failed: No active nodes left test_basic_ssl_test (security.x509tests.x509tests) ... Traceback (most recent call last): File "pytests/basetestcase.py", line 203, in setUp self._cluster_cleanup() File "pytests/basetestcase.py", line 494, in _cluster_cleanup ClusterOperationHelper.cleanup_cluster(self.servers, master=self.master) File "lib/membase/helper/cluster_helper.py", line 259, in cleanup_cluster wait_for_rebalance=wait_for_rebalance) File "lib/membase/api/rest_client.py", line 110, in remove_nodes self.rest.rebalance(knownNodes, ejectedNodes) File "lib/membase/api/rest_client.py", line 1433, in rebalance parameters=params) InvalidArgumentException: controller/rebalance with error message No active nodes left failed when invoked with parameters: password=password&ejectedNodes=ns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6&user=Administrator&knownNodes=ns_1%40cb-example-0003.cb-example.default.svc%2Cns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6 ERROR ====================================================================== ERROR: test_basic_ssl_test (security.x509tests.x509tests) ---------------------------------------------------------------------- Traceback (most recent call last): File "pytests/security/x509tests.py", line 17, in setUp super(x509tests, self).setUp() File "pytests/basetestcase.py", line 352, in setUp self.fail(e) AssertionError: controller/rebalance with error message No active nodes left failed when invoked with parameters: password=password&ejectedNodes=ns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6&user=Administrator&knownNodes=ns_1%40cb-example-0003.cb-example.default.svc%2Cns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6 ---------------------------------------------------------------------- Ran 1 test in 8.663s FAILED (errors=1) Cluster instance shutdown with force summary so far suite recreatebuckettests.RecreateMembaseBuckets , pass 0 , fail 1 failures so far... recreatebuckettests.RecreateMembaseBuckets.test_default_moxi summary so far suite deletebuckettests.DeleteMembaseBuckets , pass 1 , fail 0 summary so far suite createbuckettests.CreateMembaseBucketsTests , pass 1 , fail 4 failures so far... createbuckettests.CreateMembaseBucketsTests.test_default_moxi createbuckettests.CreateMembaseBucketsTests.test_default_on_non_default_port createbuckettests.CreateMembaseBucketsTests.test_non_default_case_sensitive_different_port createbuckettests.CreateMembaseBucketsTests.test_two_replica summary so far suite setgettests.MembaseBucket , pass 1 , fail 0 summary so far suite expirytests.ExpiryTests , pass 1 , fail 0 summary so far suite memcapable.GetlTests , pass 2 , fail 0 summary so far suite memorysanitytests.MemorySanity , pass 1 , fail 0 summary so far suite drainratetests.DrainRateTests , pass 1 , fail 0 summary so far suite view.viewquerytests.ViewQueryTests , pass 1 , fail 0 summary so far suite view.createdeleteview.CreateDeleteViewTests , pass 1 , fail 0 summary so far suite rebalance.rebalancein.RebalanceInTests , pass 0 , fail 1 failures so far... rebalance.rebalancein.RebalanceInTests.rebalance_in_with_ops summary so far suite rebalance.rebalanceout.RebalanceOutTests , pass 0 , fail 1 failures so far... rebalance.rebalanceout.RebalanceOutTests.rebalance_out_with_ops summary so far suite failover.failovertests.FailoverTests , pass 0 , fail 1 failures so far... failover.failovertests.FailoverTests.test_failover_normal summary so far suite CCCP.CCCP , pass 0 , fail 2 failures so far... CCCP.CCCP.test_get_config_client CCCP.CCCP.test_not_my_vbucket_config summary so far suite flush.bucketflush.BucketFlushTests , pass 0 , fail 1 failures so far... flush.bucketflush.BucketFlushTests.bucketflush_with_data_ops_moxi summary so far suite security.audittest.auditTest , pass 0 , fail 1 failures so far... security.audittest.auditTest.test_bucketEvents summary so far suite tuqquery.tuq_precedence.PrecedenceTests , pass 0 , fail 2 failures so far... tuqquery.tuq_precedence.PrecedenceTests.test_case_and_like tuqquery.tuq_precedence.PrecedenceTests.test_case_and_like summary so far suite tuqquery.tuq_index.QueriesViewsTests , pass 0 , fail 3 failures so far... tuqquery.tuq_index.QueriesViewsTests.test_primary_create_delete_index tuqquery.tuq_index.QueriesViewsTests.test_primary_create_delete_index tuqquery.tuq_index.QueriesViewsTests.test_explain_index_attr summary so far suite tuqquery.tuq_dml.DMLQueryTests , pass 0 , fail 1 failures so far... tuqquery.tuq_dml.DMLQueryTests.test_sanity summary so far suite 2i.indexscans_2i.SecondaryIndexingScanTests , pass 0 , fail 1 failures so far... 2i.indexscans_2i.SecondaryIndexingScanTests.test_multi_create_query_explain_drop_index summary so far suite xdcr.uniXDCR.unidirectional , pass 0 , fail 1 failures so far... xdcr.uniXDCR.unidirectional.load_with_ops summary so far suite xdcr.filterXDCR.XDCRFilterTests , pass 1 , fail 1 failures so far... xdcr.filterXDCR.XDCRFilterTests.test_xdcr_with_filter summary so far suite xdcr.biXDCR.bidirectional , pass 0 , fail 1 failures so far... xdcr.biXDCR.bidirectional.load_with_async_ops summary so far suite fts.stable_topology_fts.StableTopFTS , pass 0 , fail 1 failures so far... fts.stable_topology_fts.StableTopFTS.run_default_index_query summary so far suite ent_backup_restore.enterprise_backup_restore_test.EnterpriseBackupRestoreTest , pass 0 , fail 1 failures so far... ent_backup_restore.enterprise_backup_restore_test.EnterpriseBackupRestoreTest.test_backup_restore_sanity summary so far suite tuqquery.tuq_2i_index.QueriesIndexTests , pass 0 , fail 1 failures so far... tuqquery.tuq_2i_index.QueriesIndexTests.test_covering_index summary so far suite subdoc.subdoc_nested_dataset.SubdocNestedDataset , pass 0 , fail 1 failures so far... subdoc.subdoc_nested_dataset.SubdocNestedDataset.test_sanity summary so far suite tuqquery.tuq_advancedcbqshell.AdvancedQueryTests , pass 1 , fail 0 summary so far suite security.x509tests.x509tests , pass 0 , fail 1 failures so far... security.x509tests.x509tests.test_basic_ssl_test testrunner logs, diags and results are available under /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_39 Logs will be stored at /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_40 ./testrunner -i ./4node.ini -t 2i.indexscans_2i.SecondaryIndexingScanTests.test_multi_create_query_explain_drop_index,groups=simple,doc-per-day=10,dataset=default,gsi_type=memory_optimized Test Input params: {'doc-per-day': '10', 'logs_folder': '/testrunner/logs/testrunner-18-Jan-25_21-16-28/test_40', 'gsi_type': 'memory_optimized', 'conf_file': './4node.conf', 'dataset': 'default', 'cluster_name': '4node', 'ini': './4node.ini', 'groups': 'simple', 'case_number': 40, 'num_nodes': 4, 'spec': '4node'} [2018-01-25 22:19:44,956] - [remote_util:232] INFO - connecting to 172.17.0.8 with username:root password:couchbase ssh_key: [2018-01-25 22:19:45,040] - [remote_util:266] INFO - Connected to 172.17.0.8 [2018-01-25 22:19:45,480] - [rest_client:1964] INFO - Node version in cluster 5.0.1-5003-enterprise [2018-01-25 22:19:45,490] - [rest_client:1948] INFO - Node cb-example-0003.cb-example.default.svc not part of cluster inactiveFailed [2018-01-25 22:19:45,490] - [rest_client:1974] INFO - Node versions in cluster [u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise', u'5.0.1-5003-enterprise'] [2018-01-25 22:19:45,490] - [basetestcase:201] INFO - ============== basetestcase setup was started for test #40 test_multi_create_query_explain_drop_index============== [2018-01-25 22:19:45,504] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.8 [2018-01-25 22:19:45,504] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 22:19:47,513] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.6 [2018-01-25 22:19:47,513] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 22:19:49,522] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.7 [2018-01-25 22:19:49,522] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 22:19:51,530] - [bucket_helper:142] INFO - deleting existing buckets [] on 172.17.0.5 [2018-01-25 22:19:51,531] - [bucket_helper:168] INFO - sleep 2 seconds to make sure all buckets were deleted completely. [2018-01-25 22:19:53,548] - [cluster_helper:254] INFO - rebalancing all nodes in order to remove nodes [2018-01-25 22:19:53,551] - [rest_client:1423] INFO - rebalance params : {'password': 'password', 'ejectedNodes': u'ns_1@172.17.0.7,ns_1@172.17.0.5,ns_1@172.17.0.6', 'user': 'Administrator', 'knownNodes': u'ns_1@cb-example-0003.cb-example.default.svc,ns_1@172.17.0.7,ns_1@172.17.0.5,ns_1@172.17.0.6'} [2018-01-25 22:19:53,555] - [rest_client:807] ERROR - POST http://172.17.0.8:8091/controller/rebalance body: password=password&ejectedNodes=ns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6&user=Administrator&knownNodes=ns_1%40cb-example-0003.cb-example.default.svc%2Cns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6 headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==\n'} error: 400 reason: status: 400, content: No active nodes left No active nodes left auth: Administrator:password [2018-01-25 22:19:53,555] - [rest_client:1430] ERROR - rebalance operation failed: No active nodes left test_multi_create_query_explain_drop_index (2i.indexscans_2i.SecondaryIndexingScanTests) ... Traceback (most recent call last): File "pytests/basetestcase.py", line 203, in setUp self._cluster_cleanup() File "pytests/basetestcase.py", line 494, in _cluster_cleanup ClusterOperationHelper.cleanup_cluster(self.servers, master=self.master) File "lib/membase/helper/cluster_helper.py", line 259, in cleanup_cluster wait_for_rebalance=wait_for_rebalance) File "lib/membase/api/rest_client.py", line 110, in remove_nodes self.rest.rebalance(knownNodes, ejectedNodes) File "lib/membase/api/rest_client.py", line 1433, in rebalance parameters=params) InvalidArgumentException: controller/rebalance with error message No active nodes left failed when invoked with parameters: password=password&ejectedNodes=ns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6&user=Administrator&knownNodes=ns_1%40cb-example-0003.cb-example.default.svc%2Cns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6 ERROR ====================================================================== ERROR: test_multi_create_query_explain_drop_index (2i.indexscans_2i.SecondaryIndexingScanTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "pytests/2i/indexscans_2i.py", line 11, in setUp super(SecondaryIndexingScanTests, self).setUp() File "pytests/2i/base_2i.py", line 15, in setUp super(BaseSecondaryIndexingTests, self).setUp() File "pytests/2i/newtuq.py", line 10, in setUp super(QueryTests, self).setUp() File "pytests/basetestcase.py", line 352, in setUp self.fail(e) AssertionError: controller/rebalance with error message No active nodes left failed when invoked with parameters: password=password&ejectedNodes=ns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6&user=Administrator&knownNodes=ns_1%40cb-example-0003.cb-example.default.svc%2Cns_1%40172.17.0.7%2Cns_1%40172.17.0.5%2Cns_1%40172.17.0.6 ---------------------------------------------------------------------- Ran 1 test in 8.601s FAILED (errors=1) Cluster instance shutdown with force summary so far suite recreatebuckettests.RecreateMembaseBuckets , pass 0 , fail 1 failures so far... recreatebuckettests.RecreateMembaseBuckets.test_default_moxi summary so far suite deletebuckettests.DeleteMembaseBuckets , pass 1 , fail 0 summary so far suite createbuckettests.CreateMembaseBucketsTests , pass 1 , fail 4 failures so far... createbuckettests.CreateMembaseBucketsTests.test_default_moxi createbuckettests.CreateMembaseBucketsTests.test_default_on_non_default_port createbuckettests.CreateMembaseBucketsTests.test_non_default_case_sensitive_different_port createbuckettests.CreateMembaseBucketsTests.test_two_replica summary so far suite setgettests.MembaseBucket , pass 1 , fail 0 summary so far suite expirytests.ExpiryTests , pass 1 , fail 0 summary so far suite memcapable.GetlTests , pass 2 , fail 0 summary so far suite memorysanitytests.MemorySanity , pass 1 , fail 0 summary so far suite drainratetests.DrainRateTests , pass 1 , fail 0 summary so far suite view.viewquerytests.ViewQueryTests , pass 1 , fail 0 summary so far suite view.createdeleteview.CreateDeleteViewTests , pass 1 , fail 0 summary so far suite rebalance.rebalancein.RebalanceInTests , pass 0 , fail 1 failures so far... rebalance.rebalancein.RebalanceInTests.rebalance_in_with_ops summary so far suite rebalance.rebalanceout.RebalanceOutTests , pass 0 , fail 1 failures so far... rebalance.rebalanceout.RebalanceOutTests.rebalance_out_with_ops summary so far suite failover.failovertests.FailoverTests , pass 0 , fail 1 failures so far... failover.failovertests.FailoverTests.test_failover_normal summary so far suite CCCP.CCCP , pass 0 , fail 2 failures so far... CCCP.CCCP.test_get_config_client CCCP.CCCP.test_not_my_vbucket_config summary so far suite flush.bucketflush.BucketFlushTests , pass 0 , fail 1 failures so far... flush.bucketflush.BucketFlushTests.bucketflush_with_data_ops_moxi summary so far suite security.audittest.auditTest , pass 0 , fail 1 failures so far... security.audittest.auditTest.test_bucketEvents summary so far suite tuqquery.tuq_precedence.PrecedenceTests , pass 0 , fail 2 failures so far... tuqquery.tuq_precedence.PrecedenceTests.test_case_and_like tuqquery.tuq_precedence.PrecedenceTests.test_case_and_like summary so far suite tuqquery.tuq_index.QueriesViewsTests , pass 0 , fail 3 failures so far... tuqquery.tuq_index.QueriesViewsTests.test_primary_create_delete_index tuqquery.tuq_index.QueriesViewsTests.test_primary_create_delete_index tuqquery.tuq_index.QueriesViewsTests.test_explain_index_attr summary so far suite tuqquery.tuq_dml.DMLQueryTests , pass 0 , fail 1 failures so far... tuqquery.tuq_dml.DMLQueryTests.test_sanity summary so far suite 2i.indexscans_2i.SecondaryIndexingScanTests , pass 0 , fail 2 failures so far... 2i.indexscans_2i.SecondaryIndexingScanTests.test_multi_create_query_explain_drop_index 2i.indexscans_2i.SecondaryIndexingScanTests.test_multi_create_query_explain_drop_index summary so far suite xdcr.uniXDCR.unidirectional , pass 0 , fail 1 failures so far... xdcr.uniXDCR.unidirectional.load_with_ops summary so far suite xdcr.filterXDCR.XDCRFilterTests , pass 1 , fail 1 failures so far... xdcr.filterXDCR.XDCRFilterTests.test_xdcr_with_filter summary so far suite xdcr.biXDCR.bidirectional , pass 0 , fail 1 failures so far... xdcr.biXDCR.bidirectional.load_with_async_ops summary so far suite fts.stable_topology_fts.StableTopFTS , pass 0 , fail 1 failures so far... fts.stable_topology_fts.StableTopFTS.run_default_index_query summary so far suite ent_backup_restore.enterprise_backup_restore_test.EnterpriseBackupRestoreTest , pass 0 , fail 1 failures so far... ent_backup_restore.enterprise_backup_restore_test.EnterpriseBackupRestoreTest.test_backup_restore_sanity summary so far suite tuqquery.tuq_2i_index.QueriesIndexTests , pass 0 , fail 1 failures so far... tuqquery.tuq_2i_index.QueriesIndexTests.test_covering_index summary so far suite subdoc.subdoc_nested_dataset.SubdocNestedDataset , pass 0 , fail 1 failures so far... subdoc.subdoc_nested_dataset.SubdocNestedDataset.test_sanity summary so far suite tuqquery.tuq_advancedcbqshell.AdvancedQueryTests , pass 1 , fail 0 summary so far suite security.x509tests.x509tests , pass 0 , fail 1 failures so far... security.x509tests.x509tests.test_basic_ssl_test testrunner logs, diags and results are available under /testrunner/logs/testrunner-18-Jan-25_21-16-28/test_40 Run after suite setup for 2i.indexscans_2i.SecondaryIndexingScanTests.test_multi_create_query_explain_drop_index Thread Cluster_Thread was not properly terminated, will be terminated now. Thread Thread-127 was not properly terminated, will be terminated now. Thread Thread-423 was not properly terminated, will be terminated now. Thread Thread-117 was not properly terminated, will be terminated now. Thread Thread-400 was not properly terminated, will be terminated now. Thread Thread-432 was not properly terminated, will be terminated now. Thread Thread-406 was not properly terminated, will be terminated now. Thread Thread-426 was not properly terminated, will be terminated now. Thread Thread-31 was not properly terminated, will be terminated now. Thread Thread-28 was not properly terminated, will be terminated now. Thread Thread-25 was not properly terminated, will be terminated now. Thread Thread-22 was not properly terminated, will be terminated now. Thread Thread-40 was not properly terminated, will be terminated now. Thread Thread-417 was not properly terminated, will be terminated now. Thread Thread-16 was not properly terminated, will be terminated now. Thread Thread-34 was not properly terminated, will be terminated now. Thread Cluster_Thread was not properly terminated, will be terminated now. Thread Thread-13 was not properly terminated, will be terminated now. Thread Thread-122 was not properly terminated, will be terminated now. Thread Cluster_Thread was not properly terminated, will be terminated now. Thread Thread-43 was not properly terminated, will be terminated now. Thread Cluster_Thread was not properly terminated, will be terminated now. Thread Thread-4 was not properly terminated, will be terminated now. Thread Thread-1 was not properly terminated, will be terminated now. Thread Thread-112 was not properly terminated, will be terminated now. Thread Thread-46 was not properly terminated, will be terminated now. Thread Thread-19 was not properly terminated, will be terminated now. Thread Thread-10 was not properly terminated, will be terminated now. Thread Thread-7 was not properly terminated, will be terminated now. Thread Thread-37 was not properly terminated, will be terminated now. Thread Cluster_Thread was not properly terminated, will be terminated now. Thread Thread-429 was not properly terminated, will be terminated now.