Fix analytics failure in 1.2.x when run with MH

When we run the TestAnalyticsCreateDataSet, it fails with the following error:

— FAIL: TestOperator (1236.64s)
— FAIL: TestOperator/TestAnalyticsCreateDataSet (174.21s)
crd_util.go:26: creating couchbase cluster: test-couchbase-hpgtr
util.go:680: 2019-08-01 14:27:06.087404 -0700 PDT m=+75.768731644 Cluster nodes (0) does not match expected nodes (3)

util.go:680: 2019-08-01 14:27:16.020736 -0700 PDT m=+85.702394834 Cluster nodes (0) does not match expected nodes (3)

util.go:680: 2019-08-01 14:27:26.024833 -0700 PDT m=+95.706825577 Cluster nodes (1) does not match expected nodes (3)

util.go:680: 2019-08-01 14:27:36.018385 -0700 PDT m=+105.700711087 Cluster nodes (1) does not match expected nodes (3)

util.go:680: 2019-08-01 14:27:46.022857 -0700 PDT m=+115.705516465 Cluster nodes (1) does not match expected nodes (3)

util.go:680: 2019-08-01 14:27:56.019308 -0700 PDT m=+125.702301186 Cluster nodes (2) does not match expected nodes (3)

util.go:680: 2019-08-01 14:28:06.016084 -0700 PDT m=+135.699410769 Cluster nodes (2) does not match expected nodes (3)

util.go:680: 2019-08-01 14:28:16.022912 -0700 PDT m=+145.706572095 Cluster nodes (2) does not match expected nodes (3)

util.go:680: 2019-08-01 14:28:26.015428 -0700 PDT m=+155.699421680 Cluster nodes (2) does not match expected nodes (3)

util.go:680: 2019-08-01 14:28:36.015073 -0700 PDT m=+165.699400133 Cluster Status Conditions: (map[Available:0xc0002df040 Balanced:0xc0002df090 Scaling:0xc0002df0e0])
util.go:680: 2019-08-01 14:28:36.015169 -0700 PDT m=+165.699495907 Cluster scaling ...
util.go:680: 2019-08-01 14:28:46.014908 -0700 PDT m=+175.699568518 Cluster Status Conditions: (map[Available:0xc0004bd270 Balanced:0xc0004bd2c0 Scaling:0xc0004bd310])
util.go:680: 2019-08-01 14:28:46.014987 -0700 PDT m=+175.699646899 Cluster scaling ...
util.go:680: 2019-08-01 14:28:56.015302 -0700 PDT m=+185.700295667 Cluster Status Conditions: (map[Available:0xc0002df220 Balanced:0xc0002df2c0 Scaling:0xc0002df310])
util.go:680: 2019-08-01 14:28:56.0154 -0700 PDT m=+185.700394322 Cluster scaling ...
util.go:680: 2019-08-01 14:29:06.016136 -0700 PDT m=+195.701463060 Cluster Status Conditions: (map[Available:0xc0000fc1e0 Balanced:0xc0000fc280 Scaling:0xc0000fc2d0])
util.go:680: 2019-08-01 14:29:06.016225 -0700 PDT m=+195.701552025 Cluster unbalanced ...
util.go:680: 2019-08-01 14:29:16.020954 -0700 PDT m=+205.706614993 Cluster Status Conditions: (map[Available:0xc0000fc410 Balanced:0xc0000fc460 Scaling:0xc0000fc4b0])
util.go:680: 2019-08-01 14:29:16.021066 -0700 PDT m=+205.706727027 Cluster scaling ...
util.go:680: 2019-08-01 14:29:26.014942 -0700 PDT m=+215.700936385 Cluster Status Conditions: (map[Available:0xc0000fc640 Balanced:0xc0000fc690 Scaling:0xc0000fc730])
util.go:680: 2019-08-01 14:29:26.015028 -0700 PDT m=+215.701022762 Cluster scaling ...
util.go:680: 2019-08-01 14:29:36.009957 -0700 PDT m=+225.696285042 Cluster Status Conditions: (map[Available:0xc000ab00a0 Balanced:0xc000ab00f0])
util.go:680: 2019-08-01 14:29:36.010037 -0700 PDT m=+225.696364387 Cluster healthy
analytics_test.go:84: CREATE DATASET testDataset1 ON `defBucket`
analytics_test.go:86: Remote call failed with response: 400 Bad Request-{
"requestID": "049c860f-acd8-4187-a6d8-c5ef30e097ad",
"errors": [

{ "code": 21004, "msg": "No statement provided" }

],
"status": "fatal",
"metrics":

{ "elapsedTime": "12.289178ms", "executionTime": "0ns", "resultCount": 0, "resultSize": 0, "processedObjects": 0, "errorCount": 1 }

}

util.go:571: Attempting to delete: [test-couchbase-hpgtr]
util.go:575: Successfully deleted: [test-couchbase-hpgtr]
util.go:585: Killing pods: [test-couchbase-hpgtr-0000 test-couchbase-hpgtr-0001 test-couchbase-hpgtr-0002]
— FAIL: TestOperator/TestAnalyticsCreateDataSet#01 (165.51s)
crd_util.go:26: creating couchbase cluster: test-couchbase-225lx
util.go:680: 2019-08-01 14:34:44.872707 -0700 PDT m=+534.569336189 Cluster nodes (0) does not match expected nodes (3)

util.go:680: 2019-08-01 14:34:54.860889 -0700 PDT m=+544.557852007 Cluster nodes (0) does not match expected nodes (3)

util.go:680: 2019-08-01 14:35:04.943424 -0700 PDT m=+554.640723143 Cluster nodes (0) does not match expected nodes (3)

util.go:680: 2019-08-01 14:35:14.862682 -0700 PDT m=+564.560311780 Cluster nodes (1) does not match expected nodes (3)

util.go:680: 2019-08-01 14:35:24.858434 -0700 PDT m=+574.556396969 Cluster nodes (1) does not match expected nodes (3)

util.go:680: 2019-08-01 14:35:34.85576 -0700 PDT m=+584.554056389 Cluster nodes (1) does not match expected nodes (3)

util.go:680: 2019-08-01 14:35:44.872653 -0700 PDT m=+594.571283756 Cluster nodes (2) does not match expected nodes (3)

util.go:680: 2019-08-01 14:35:54.856863 -0700 PDT m=+604.555826258 Cluster nodes (2) does not match expected nodes (3)

util.go:680: 2019-08-01 14:36:04.855827 -0700 PDT m=+614.555124602 Cluster nodes (2) does not match expected nodes (3)

util.go:680: 2019-08-01 14:36:14.856896 -0700 PDT m=+624.556526817 Cluster nodes (2) does not match expected nodes (3)

util.go:680: 2019-08-01 14:36:24.857612 -0700 PDT m=+634.557576742 Cluster Status Conditions: (map[Available:0xc0000fc280 Balanced:0xc0000fc2d0 Scaling:0xc0000fc320])
util.go:680: 2019-08-01 14:36:24.857743 -0700 PDT m=+634.557707631 Cluster scaling ...
util.go:680: 2019-08-01 14:36:34.85579 -0700 PDT m=+644.556088049 Cluster Status Conditions: (map[Available:0xc000a3e190 Balanced:0xc000a3e1e0 Scaling:0xc000a3e230])
util.go:680: 2019-08-01 14:36:34.855916 -0700 PDT m=+644.556213645 Cluster scaling ...
util.go:680: 2019-08-01 14:36:44.852799 -0700 PDT m=+654.553429754 Cluster Status Conditions: (map[Available:0xc000a3e3c0 Balanced:0xc000a3e410 Scaling:0xc000a3e460])
util.go:680: 2019-08-01 14:36:44.852889 -0700 PDT m=+654.553520084 Cluster unbalanced ...
util.go:680: 2019-08-01 14:36:54.858206 -0700 PDT m=+664.559170805 Cluster Status Conditions: (map[Available:0xc0000fc500 Balanced:0xc0000fc550 Scaling:0xc0000fc5a0])
util.go:680: 2019-08-01 14:36:54.858319 -0700 PDT m=+664.559284364 Cluster unbalanced ...
util.go:680: 2019-08-01 14:37:04.85486 -0700 PDT m=+674.556158294 Cluster Status Conditions: (map[Available:0xc0000fc730 Balanced:0xc0000fc780])
util.go:680: 2019-08-01 14:37:04.854971 -0700 PDT m=+674.556269183 Cluster healthy
analytics_test.go:84: CREATE DATASET testDataset1 ON `defBucket`
analytics_test.go:86: Remote call failed with response: 400 Bad Request-{
"requestID": "41d4b94b-7399-470e-90f7-9a8ba624f18a",
"errors": [

{ "code": 21004, "msg": "No statement provided" }

],
"status": "fatal",
"metrics":

{ "elapsedTime": "11.992079ms", "executionTime": "0ns", "resultCount": 0, "resultSize": 0, "processedObjects": 0, "errorCount": 1 }

}

util.go:571: Attempting to delete: [test-couchbase-225lx]
util.go:575: Successfully deleted: [test-couchbase-225lx]
util.go:585: Killing pods: [test-couchbase-225lx-0000 test-couchbase-225lx-0001 test-couchbase-225lx-0002]
— PASS: TestOperator/TestEventingCreateEventingCluster (200.38s)
crd_util.go:26: creating couchbase cluster: test-couchbase-2tfc7
util.go:680: 2019-08-01 14:43:03.203772 -0700 PDT m=+1032.917022627 Cluster nodes (0) does not match expected nodes (3)

util.go:680: 2019-08-01 14:43:13.208088 -0700 PDT m=+1042.921672349 Cluster nodes (0) does not match expected nodes (3)

util.go:680: 2019-08-01 14:43:23.206549 -0700 PDT m=+1052.920466363 Cluster nodes (1) does not match expected nodes (3)

util.go:680: 2019-08-01 14:43:33.20357 -0700 PDT m=+1062.917821130 Cluster nodes (1) does not match expected nodes (3)

util.go:680: 2019-08-01 14:43:43.204497 -0700 PDT m=+1072.919081894 Cluster nodes (1) does not match expected nodes (3)

util.go:680: 2019-08-01 14:43:53.20787 -0700 PDT m=+1082.922788412 Cluster nodes (1) does not match expected nodes (3)

util.go:680: 2019-08-01 14:44:03.203376 -0700 PDT m=+1092.918627340 Cluster nodes (1) does not match expected nodes (3)

util.go:680: 2019-08-01 14:44:13.203292 -0700 PDT m=+1102.918877101 Cluster nodes (1) does not match expected nodes (3)

util.go:680: 2019-08-01 14:44:23.205025 -0700 PDT m=+1112.920943807 Cluster nodes (2) does not match expected nodes (3)

util.go:680: 2019-08-01 14:44:33.202753 -0700 PDT m=+1122.919004771 Cluster nodes (2) does not match expected nodes (3)

util.go:680: 2019-08-01 14:44:43.204819 -0700 PDT m=+1132.921404645 Cluster nodes (2) does not match expected nodes (3)

util.go:680: 2019-08-01 14:44:53.201836 -0700 PDT m=+1142.918754767 Cluster Status Conditions: (map[Available:0xc0004bcd70 Balanced:0xc0004bcdc0 Scaling:0xc0004bce10])
util.go:680: 2019-08-01 14:44:53.201927 -0700 PDT m=+1142.918846149 Cluster scaling ...
util.go:680: 2019-08-01 14:45:03.214297 -0700 PDT m=+1152.931550300 Cluster Status Conditions: (map[Available:0xc00052bcc0 Balanced:0xc00052bd10 Scaling:0xc00052bd60])
util.go:680: 2019-08-01 14:45:03.214402 -0700 PDT m=+1152.931654744 Cluster unbalanced ...
util.go:680: 2019-08-01 14:45:13.209209 -0700 PDT m=+1162.926795485 Cluster Status Conditions: (map[Available:0xc000a3e7d0 Balanced:0xc000a3e820 Scaling:0xc000a3e870])
util.go:680: 2019-08-01 14:45:13.20935 -0700 PDT m=+1162.926936615 Cluster scaling ...
util.go:680: 2019-08-01 14:45:23.199906 -0700 PDT m=+1172.917825835 Cluster Status Conditions: (map[Available:0xc0004bd040 Balanced:0xc0004bd090 Scaling:0xc0004bd0e0])
util.go:680: 2019-08-01 14:45:23.199976 -0700 PDT m=+1172.917895914 Cluster scaling ...
util.go:680: 2019-08-01 14:45:33.206585 -0700 PDT m=+1182.924838120 Cluster Status Conditions: (map[Available:0xc0004bd220 Balanced:0xc0004bd270 Scaling:0xc0004bd2c0])
util.go:680: 2019-08-01 14:45:33.206688 -0700 PDT m=+1182.924941411 Cluster scaling ...
util.go:680: 2019-08-01 14:45:43.202684 -0700 PDT m=+1192.921270685 Cluster Status Conditions: (map[Available:0xc000d70050 Balanced:0xc000d700a0])
util.go:680: 2019-08-01 14:45:43.202764 -0700 PDT m=+1192.921350643 Cluster healthy
util.go:571: Attempting to delete: [test-couchbase-2tfc7]
util.go:575: Successfully deleted: [test-couchbase-2tfc7]
util.go:585: Killing pods: [test-couchbase-2tfc7-0000 test-couchbase-2tfc7-0001 test-couchbase-2tfc7-0002]
test_util.go:46: Suite Test Results:
test_util.go:75: 1: TestAnalyticsCreateDataSet...FAIL
test_util.go:73: 2: TestEventingCreateEventingCluster...PASS
test_util.go:99: Failures:
test_util.go:101: 1: TestAnalyticsCreateDataSet
test_util.go:112:
Pass: 1.000000
Fail: 1.000000
Pass Rate: 50.000000
test_util.go:124: suite contains failures
FAIL
time="2019-08-01T14:46:27-07:00" level=info msg="Cleaning up namespace: default"
time="2019-08-01T14:46:27-07:00" level=info msg="Deleting jobs"
time="2019-08-01T14:46:42-07:00" level=info msg="Deleting Couchbase clusters"
time="2019-08-01T14:46:42-07:00" level=info msg="Deleting Couchbase services"
time="2019-08-01T14:46:42-07:00" level=info msg="Deleting jobs"
time="2019-08-01T14:46:47-07:00" level=info msg="Deleting Couchbase clusters"
time="2019-08-01T14:46:48-07:00" level=info msg="Deleting Couchbase services"
time="2019-08-01T14:46:48-07:00" level=info msg="Namespace cleaned-up successfully"
FAIL github.com/couchbase/couchbase-operator/test/e2e 1257.904s

Looks like the way we are creating dataset does not work with MH - attaching logs and console output