Building

Started building at 2019/12/10 20:06:25
Using pegged server, 4928 build
Calculating base
Updating mirror
Basing run on 6.5.0-4928 88475de10
Updating tree for run 10.12.2019-20.06
query is at 07e8269, changes since last good build: none
gometa is at 563cdf3, changes since last good build: none
ns_server is at 3327aac, changes since last good build: none
couchstore is at fff3e20, changes since last good build: none
forestdb is at 4c3b2f9, changes since last good build: none
kv_engine is at 9029b44, changes since last good build: none
Switching indexing to unstable
indexing is at 2d51e4d, changes since last good build: none
Switching plasma to unstable
plasma is at ea859cb, changes since last good build: none
Switching nitro to unstable
nitro is at bf47148, changes since last good build: none
Switching gometa to master
gometa is at 563cdf3, changes since last good build: none
Switching testrunner to master
testrunner is at dda5bad, changes since last good build: none
Pulling in uncommitted change 118135 at refs/changes/35/118135/7
Total 12 (delta 0), reused 4 (delta 0)
[unstable 06d23f41] MB-36524 Add alter index tests
 Author: Varun Velamuri 
 Date: Tue Nov 19 15:51:43 2019 +0530
 4 files changed, 223 insertions(+), 2 deletions(-)
 create mode 100644 secondary/tests/functionaltests/setn_alter_test.go
Building cmakefiles and deps
Building main product

... A new repo command ( 1.26) is available.
... You should upgrade soon:

    cp /opt/build/.repo/repo/repo /home/buildbot/bin/repo


... A new repo command ( 1.26) is available.
... You should upgrade soon:

    cp /opt/build/.repo/repo/repo /home/buildbot/bin/repo

Testing

Started testing at 2019/12/10 20:15:17
Testing mode: sanity,unit,functional,integration
Using storage type: fdb

Simple Test

Dec 10 20:19:27 rebalance_in_with_ops (rebalance.rebalancein.RebalanceInTests) ... ok
Dec 10 20:22:27 rebalance_in_with_ops (rebalance.rebalancein.RebalanceInTests) ... ok
Dec 10 20:23:09 do_warmup_100k (memcapable.WarmUpMemcachedTest) ... ok
Dec 10 20:23:21 summary so far suite rebalance.rebalancein.RebalanceInTests , pass 1 , fail 0
Dec 10 20:23:21 summary so far suite rebalance.rebalancein.RebalanceInTests , pass 2 , fail 0
Dec 10 20:23:21 summary so far suite rebalance.rebalancein.RebalanceInTests , pass 2 , fail 0
Dec 10 20:23:21 summary so far suite memcapable.WarmUpMemcachedTest , pass 1 , fail 0
Dec 10 20:24:23 {'cluster_name': 'dev-4-nodes-xdcr', 'ddoc_ops': 'create', 'log_level': 'CRITICAL', 'num_ddocs': '1', 'items': '1000', 'test_with_view': 'True', 'makefile': 'True', 'num_views_per_ddoc': '10', 'conf_file': 'conf/simple.conf', 'skip_cleanup': 'False', 'ini': 'b/resources/devok
Dec 10 20:33:17 test_employee_dataset_startkey_endkey_queries_rebalance_in (view.viewquerytests.ViewQueryTests) ... ok
Dec 10 20:33:53 test_simple_dataset_stale_queries_data_modification (view.viewquerytests.ViewQueryTests) ... ok
Dec 10 20:36:55 load_with_ops (xdcr.uniXDCR.unidirectional) ... ok
Dec 10 20:37:53 load_with_failover (xdcr.uniXDCR.unidirectional) ... -4-nodes-xdcr.ini', 'case_number': 4, 'get-logs-cluster-run': 'True', 'num_nodes': 4, 'logs_folder': '/opt/build/testrunner/logs/testrunner-19-Dec-10_20-15-38/test_4', 'spec': 'simple'}
Dec 10 20:37:53 summary so far suite rebalance.rebalancein.RebalanceInTests , pass 2 , fail 0
Dec 10 20:37:53 summary so far suite memcapable.WarmUpMemcachedTest , pass 1 , fail 0
Dec 10 20:37:53 summary so far suite view.createdeleteview.CreateDeleteViewTests , pass 1 , fail 0
Dec 10 20:37:53 summary so far suite rebalance.rebalancein.RebalanceInTests , pass 2 , fail 0
Dec 10 20:37:53 summary so far suite memcapable.WarmUpMemcachedTest , pass 1 , fail 0
Dec 10 20:37:53 summary so far suite view.createdeleteview.CreateDeleteViewTests , pass 1 , fail 0
Dec 10 20:37:53 summary so far suite view.viewquerytests.ViewQueryTests , pass 1 , fail 0
Dec 10 20:37:53 summary so far suite rebalance.rebalancein.RebalanceInTests , pass 2 , fail 0
Dec 10 20:37:53 summary so far suite memcapable.WarmUpMemcachedTest , pass 1 , fail 0
Dec 10 20:37:53 summary so far suite view.createdeleteview.CreateDeleteViewTests , pass 1 , fail 0
Dec 10 20:37:53 summary so far suite view.viewquerytests.ViewQueryTests , pass 2 , fail 0
Dec 10 20:40:10 Cluster instance shutdowok
Dec 10 20:40:11 summary so far suite rebalance.rebalancein.RebalanceInTests , pass 2 , fail 0
Dec 10 20:40:11 summary so far suite memcapable.WarmUpMemcachedTest , pass 1 , fail 0
Dec 10 20:40:11 summary so far suite view.createdeleteview.CreateDeleteViewTests , pass 1 , fail 0
Dec 10 20:40:11 summary so far suite view.viewquerytests.ViewQueryTests , pass 2 , fail 0
Dec 10 20:40:11 summary so far suite xdcr.uniXDCR.unidirectional , pass 1 , fail 0
Dec 10 20:40:11 ./testrunner -i b/resources/dev-4-nodes-xdcr.ini -p makefile=True,log_level=CRITICAL -t xdcr.uniXDCR.unidirectional.load_with_failover,replicas=1,items=10000,ctopology=chain,rdirection=unidirection,doc-ops=update-delete,failover=source,get-logs-cluster-run=True
Dec 10 20:40:11 {'doc-ops': 'update-delete', 'log_level': 'CRITICAL', 'replicas': '1', 'items': '10000', 'failover': 'source', 'makefile': 'True', 'conf_file': 'conf/simple.conf', 'num_nodes': 4, 'cluster_name': 'dev-4-nodes-xdcr', 'ctopology': 'chain', 'rdirection': 'unidirection', 'ini': 'b/resources/dev-4-nodes-xdcr.ini', 'case_number': 8, 'get-logs-cluster-run': 'True', 'logs_folder': '/opt/build/testrunner/logs/testrunner-19-Dec-10_20-15-38/test_8', 'spec': 'simple'}
Dec 10 20:40:11 summary so far suite rebalance.rebalancein.RebalanceInTests , pass 2 , fail 0
Dec 10 20:40:11 summary so far suite memcapable.WarmUpMemcachedTest , pass 1 , fail 0
Dec 10 20:40:11 summary so far suite view.createdeleteview.CreateDeleteViewTests , pass 1 , fail 0
Dec 10 20:40:11 summary so far suite view.viewquerytests.ViewQueryTests , pass 2 , fail 0
Dec 10 20:40:11 summary so far suite xdcr.uniXDCR.unidirectional , pass 2 , fail 0
Dec 10 20:40:11 Run after suite setup for xdcr.uniXDCR.unidirectional.load_with_failover
Dec 10 20:40:11 rebalance.rebalancein.RebalanceInTests.rebalance_in_with_ops  pass
Dec 10 20:40:11 rebalance.rebalancein.RebalanceInTests.rebalance_in_with_ops  pass
Dec 10 20:40:11 memcapable.WarmUpMemcachedTest.do_warmup_100k  pass
Dec 10 20:40:11 view.createdeleteview.CreateDeleteViewTests.test_view_ops  pass
Dec 10 20:40:11 view.viewquerytests.ViewQueryTests.test_employee_dataset_startkey_endkey_queries_rebalance_in  pass
Dec 10 20:40:11 view.viewquerytests.ViewQueryTests.test_simple_dataset_stale_queries_data_modification  pass
Dec 10 20:40:11 xdcr.uniXDCR.unidirectional.load_with_ops  pass
Dec 10 20:40:11 xdcr.uniXDCR.unidirectional.load_with_failover  pass

Unit tests

=== RUN   TestInteger
--- PASS: TestInteger (0.00s)
=== RUN   TestSmallDecimal
--- PASS: TestSmallDecimal (0.00s)
=== RUN   TestLargeDecimal
--- PASS: TestLargeDecimal (0.00s)
=== RUN   TestFloat
--- PASS: TestFloat (0.00s)
=== RUN   TestSuffixCoding
--- PASS: TestSuffixCoding (0.00s)
=== RUN   TestCodecLength
--- PASS: TestCodecLength (0.00s)
=== RUN   TestSpecialString
--- PASS: TestSpecialString (0.00s)
=== RUN   TestCodecNoLength
--- PASS: TestCodecNoLength (0.00s)
=== RUN   TestCodecJSON
--- PASS: TestCodecJSON (0.00s)
=== RUN   TestReference
--- PASS: TestReference (0.00s)
=== RUN   TestN1QLEncode
--- PASS: TestN1QLEncode (0.00s)
=== RUN   TestArrayExplodeJoin
--- PASS: TestArrayExplodeJoin (0.00s)
=== RUN   TestN1QLDecode
--- PASS: TestN1QLDecode (0.00s)
=== RUN   TestN1QLDecode2
--- PASS: TestN1QLDecode2 (0.00s)
=== RUN   TestArrayExplodeJoin2
--- PASS: TestArrayExplodeJoin2 (0.00s)
=== RUN   TestMB28956
--- PASS: TestMB28956 (0.00s)
=== RUN   TestFixEncodedInt
--- PASS: TestFixEncodedInt (0.00s)
=== RUN   TestN1QLDecodeLargeInt64
--- PASS: TestN1QLDecodeLargeInt64 (0.00s)
=== RUN   TestMixedModeFixEncodedInt
TESTING [4111686018427387900, -8223372036854775808, 822337203685477618] 
PASS 
TESTING [0] 
PASS 
TESTING [0.0] 
PASS 
TESTING [0.0000] 
PASS 
TESTING [0.0000000] 
PASS 
TESTING [-0] 
PASS 
TESTING [-0.0] 
PASS 
TESTING [-0.0000] 
PASS 
TESTING [-0.0000000] 
PASS 
TESTING [1] 
PASS 
TESTING [20] 
PASS 
TESTING [3456] 
PASS 
TESTING [7645000] 
PASS 
TESTING [9223372036854775807] 
PASS 
TESTING [9223372036854775806] 
PASS 
TESTING [9223372036854775808] 
PASS 
TESTING [92233720368547758071234000] 
PASS 
TESTING [92233720368547758071234987437653] 
PASS 
TESTING [12300000000000000000000000000000056] 
PASS 
TESTING [12300000000000000000000000000000000] 
PASS 
TESTING [123000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000056] 
PASS 
TESTING [123000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000] 
PASS 
TESTING [12300000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000056] 
PASS 
TESTING [210690] 
PASS 
TESTING [90000] 
PASS 
TESTING [123000000] 
PASS 
TESTING [3.60e2] 
PASS 
TESTING [36e2] 
PASS 
TESTING [1.9999999999e10] 
PASS 
TESTING [1.99999e10] 
PASS 
TESTING [1.99999e5] 
PASS 
TESTING [0.00000000000012e15] 
PASS 
TESTING [7.64507352e8] 
PASS 
TESTING [9.2233720368547758071234987437653e31] 
PASS 
TESTING [2650e-1] 
PASS 
TESTING [26500e-1] 
PASS 
TESTING [-1] 
PASS 
TESTING [-20] 
PASS 
TESTING [-3456] 
PASS 
TESTING [-7645000] 
PASS 
TESTING [-9223372036854775808] 
PASS 
TESTING [-9223372036854775807] 
PASS 
TESTING [-9223372036854775806] 
PASS 
TESTING [-9223372036854775809] 
PASS 
TESTING [-92233720368547758071234000] 
PASS 
TESTING [-92233720368547758071234987437653] 
PASS 
TESTING [-12300000000000000000000000000000056] 
PASS 
TESTING [-12300000000000000000000000000000000] 
PASS 
TESTING [-123000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000056] 
PASS 
TESTING [-123000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000] 
PASS 
TESTING [-210690] 
PASS 
TESTING [-90000] 
PASS 
TESTING [-123000000] 
PASS 
TESTING [-3.60e2] 
PASS 
TESTING [-36e2] 
PASS 
TESTING [-1.9999999999e10] 
PASS 
TESTING [-1.99999e10] 
PASS 
TESTING [-1.99999e5] 
PASS 
TESTING [-0.00000000000012e15] 
PASS 
TESTING [-2650e-1] 
PASS 
TESTING [-26500e-1] 
PASS 
TESTING [0.03] 
PASS 
TESTING [198.60] 
PASS 
TESTING [2000045.178] 
PASS 
TESTING [1.7976931348623157e+308] 
PASS 
TESTING [0.000000000000000000890] 
PASS 
TESTING [257953786.9864236576] 
PASS 
TESTING [257953786.9864236576e8] 
PASS 
TESTING [36.912e3] 
PASS 
TESTING [2761.67e0] 
PASS 
TESTING [2761.67e00] 
PASS 
TESTING [2761.67e000] 
PASS 
TESTING [7676546.67e-3] 
PASS 
TESTING [-0.03] 
PASS 
TESTING [-198.60] 
PASS 
TESTING [-2000045.178] 
PASS 
TESTING [-1.7976931348623157e+308] 
PASS 
TESTING [-0.000000000000000000890] 
PASS 
TESTING [-257953786.9864236576] 
PASS 
TESTING [-257953786.9864236576e8] 
PASS 
TESTING [-36.912e3] 
PASS 
TESTING [-2761.67e0] 
PASS 
TESTING [-2761.67e00] 
PASS 
TESTING [-2761.67e000] 
PASS 
TESTING [-7676546.67e-3] 
PASS 
--- PASS: TestMixedModeFixEncodedInt (0.01s)
=== RUN   TestCodecDesc
--- PASS: TestCodecDesc (0.00s)
=== RUN   TestCodecDescPropLen
--- PASS: TestCodecDescPropLen (0.00s)
=== RUN   TestCodecDescSplChar
--- PASS: TestCodecDescSplChar (0.00s)
PASS
ok  	github.com/couchbase/indexing/secondary/collatejson	0.040s
=== RUN   TestForestDBIterator
2019-12-10T20:40:28.700+05:30 [INFO][FDB] Forestdb blockcache size 134217728 initialized in 5455 us

2019-12-10T20:40:28.701+05:30 [INFO][FDB] Forestdb opened database file test
2019-12-10T20:40:28.705+05:30 [INFO][FDB] Forestdb closed database file test
--- PASS: TestForestDBIterator (0.01s)
=== RUN   TestForestDBIteratorSeek
2019-12-10T20:40:28.705+05:30 [INFO][FDB] Forestdb opened database file test
2019-12-10T20:40:28.708+05:30 [INFO][FDB] Forestdb closed database file test
--- PASS: TestForestDBIteratorSeek (0.00s)
=== RUN   TestPrimaryIndexEntry
--- PASS: TestPrimaryIndexEntry (0.00s)
=== RUN   TestSecondaryIndexEntry
--- PASS: TestSecondaryIndexEntry (0.00s)
=== RUN   TestPrimaryIndexEntryMatch
--- PASS: TestPrimaryIndexEntryMatch (0.00s)
=== RUN   TestSecondaryIndexEntryMatch
--- PASS: TestSecondaryIndexEntryMatch (0.01s)
=== RUN   TestLongDocIdEntry
--- PASS: TestLongDocIdEntry (0.00s)
=== RUN   TestMemDBInsertionPerf
Initial build: 10000000 items took 2m4.680350616s -> 80205.10008669096 items/s
Incr build: 10000000 items took 2m0.95043801s -> 82678.49347658597 items/s
Main Index: {
"node_count":             18000000,
"soft_deletes":           0,
"read_conflicts":         0,
"insert_conflicts":       3,
"next_pointers_per_node": 1.3333,
"memory_used":            1695887708,
"node_allocs":            18000000,
"node_frees":             0,
"level_node_distribution":{
"level0": 13500190,
"level1": 3375550,
"level2": 843073,
"level3": 210794,
"level4": 52734,
"level5": 13348,
"level6": 3245,
"level7": 783,
"level8": 220,
"level9": 42,
"level10": 17,
"level11": 3,
"level12": 1,
"level13": 0,
"level14": 0,
"level15": 0,
"level16": 0,
"level17": 0,
"level18": 0,
"level19": 0,
"level20": 0,
"level21": 0,
"level22": 0,
"level23": 0,
"level24": 0,
"level25": 0,
"level26": 0,
"level27": 0,
"level28": 0,
"level29": 0,
"level30": 0,
"level31": 0,
"level32": 0
}
}
Back Index 0 : {
"FastHTCount":  625000,
"SlowHTCount":  0,
"Conflicts":   0,
"MemoryInUse": 26250000
}
Back Index 1 : {
"FastHTCount":  625000,
"SlowHTCount":  0,
"Conflicts":   0,
"MemoryInUse": 26250000
}
Back Index 2 : {
"FastHTCount":  625000,
"SlowHTCount":  0,
"Conflicts":   0,
"MemoryInUse": 26250000
}
Back Index 3 : {
"FastHTCount":  625000,
"SlowHTCount":  0,
"Conflicts":   0,
"MemoryInUse": 26250000
}
Back Index 4 : {
"FastHTCount":  625000,
"SlowHTCount":  0,
"Conflicts":   0,
"MemoryInUse": 26250000
}
Back Index 5 : {
"FastHTCount":  625000,
"SlowHTCount":  0,
"Conflicts":   0,
"MemoryInUse": 26250000
}
Back Index 6 : {
"FastHTCount":  625000,
"SlowHTCount":  0,
"Conflicts":   0,
"MemoryInUse": 26250000
}
Back Index 7 : {
"FastHTCount":  625000,
"SlowHTCount":  0,
"Conflicts":   0,
"MemoryInUse": 26250000
}
Back Index 8 : {
"FastHTCount":  625000,
"SlowHTCount":  0,
"Conflicts":   0,
"MemoryInUse": 26250000
}
Back Index 9 : {
"FastHTCount":  625000,
"SlowHTCount":  0,
"Conflicts":   0,
"MemoryInUse": 26250000
}
Back Index 10 : {
"FastHTCount":  625000,
"SlowHTCount":  0,
"Conflicts":   0,
"MemoryInUse": 26250000
}
Back Index 11 : {
"FastHTCount":  625000,
"SlowHTCount":  0,
"Conflicts":   0,
"MemoryInUse": 26250000
}
Back Index 12 : {
"FastHTCount":  625000,
"SlowHTCount":  0,
"Conflicts":   0,
"MemoryInUse": 26250000
}
Back Index 13 : {
"FastHTCount":  625000,
"SlowHTCount":  0,
"Conflicts":   0,
"MemoryInUse": 26250000
}
Back Index 14 : {
"FastHTCount":  625000,
"SlowHTCount":  0,
"Conflicts":   0,
"MemoryInUse": 26250000
}
Back Index 15 : {
"FastHTCount":  625000,
"SlowHTCount":  0,
"Conflicts":   0,
"MemoryInUse": 26250000
}
--- PASS: TestMemDBInsertionPerf (245.64s)
=== RUN   TestBasicsA
--- PASS: TestBasicsA (0.00s)
=== RUN   TestSizeA
--- PASS: TestSizeA (0.00s)
=== RUN   TestSizeWithFreelistA
--- PASS: TestSizeWithFreelistA (0.00s)
=== RUN   TestDequeueUptoSeqnoA
--- PASS: TestDequeueUptoSeqnoA (0.10s)
=== RUN   TestDequeueA
--- PASS: TestDequeueA (1.20s)
=== RUN   TestMultipleVbucketsA
--- PASS: TestMultipleVbucketsA (0.00s)
=== RUN   TestDequeueUptoFreelistA
--- PASS: TestDequeueUptoFreelistA (0.00s)
=== RUN   TestDequeueUptoFreelistMultVbA
--- PASS: TestDequeueUptoFreelistMultVbA (0.00s)
=== RUN   TestConcurrentEnqueueDequeueA
--- PASS: TestConcurrentEnqueueDequeueA (0.00s)
=== RUN   TestConcurrentEnqueueDequeueA1
--- PASS: TestConcurrentEnqueueDequeueA1 (10.00s)
=== RUN   TestEnqueueAppCh
--- PASS: TestEnqueueAppCh (2.00s)
PASS
ok  	github.com/couchbase/indexing/secondary/indexer	259.731s
=== RUN   TestConnPoolBasicSanity
2019-12-10T20:44:52.203+05:30 [Info] [Queryport-connpool:127.0.0.1:15151] started poolsize 3 overflow 6 low WM 3 relConn batch size 1 ...
2019-12-10T20:44:52.409+05:30 [Info] [Queryport-connpool:127.0.0.1:15151] ... stopped
2019-12-10T20:44:53.204+05:30 [Info] [Queryport-connpool:127.0.0.1:15151] Stopping releaseConnsRoutine
--- PASS: TestConnPoolBasicSanity (5.00s)
=== RUN   TestConnRelease
2019-12-10T20:44:57.205+05:30 [Info] [Queryport-connpool:127.0.0.1:15151] started poolsize 500 overflow 10 low WM 40 relConn batch size 10 ...
Waiting for connections to get released
Waiting for more connections to get released
Waiting for further more connections to get released
2019-12-10T20:45:36.940+05:30 [Info] [Queryport-connpool:127.0.0.1:15151] ... stopped
2019-12-10T20:45:37.218+05:30 [Info] [Queryport-connpool:127.0.0.1:15151] Stopping releaseConnsRoutine
--- PASS: TestConnRelease (43.74s)
=== RUN   TestLongevity
2019-12-10T20:45:40.941+05:30 [Info] [Queryport-connpool:127.0.0.1:15151] started poolsize 500 overflow 10 low WM 40 relConn batch size 10 ...
Releasing 1 conns.
Getting 2 conns.
Releasing 2 conns.
Getting 4 conns.
Releasing 1 conns.
Getting 3 conns.
Releasing 0 conns.
Getting 0 conns.
Releasing 1 conns.
Getting 0 conns.
Releasing 4 conns.
Getting 1 conns.
Releasing 2 conns.
Getting 4 conns.
Releasing 3 conns.
Getting 4 conns.
Releasing 1 conns.
Getting 0 conns.
Releasing 2 conns.
Getting 1 conns.
Releasing 0 conns.
Getting 1 conns.
Releasing 3 conns.
Getting 3 conns.
Releasing 2 conns.
Getting 2 conns.
Releasing 2 conns.
Getting 3 conns.
Releasing 0 conns.
Getting 0 conns.
2019-12-10T20:46:19.401+05:30 [Info] [Queryport-connpool:127.0.0.1:15151] ... stopped
2019-12-10T20:46:19.952+05:30 [Info] [Queryport-connpool:127.0.0.1:15151] Stopping releaseConnsRoutine
--- PASS: TestLongevity (42.46s)
=== RUN   TestSustainedHighConns
2019-12-10T20:46:23.402+05:30 [Info] [Queryport-connpool:127.0.0.1:15151] started poolsize 500 overflow 10 low WM 40 relConn batch size 10 ...
cp.curActConns = 0
Allocating 16 Connections
Returning 3 Connections
cp.curActConns = 11
Returning 2 Connections
cp.curActConns = 11
Returning 1 Connections
Allocating 4 Connections
cp.curActConns = 14
Returning 1 Connections
Allocating 12 Connections
cp.curActConns = 22
Returning 1 Connections
cp.curActConns = 24
Returning 0 Connections
Allocating 1 Connections
cp.curActConns = 25
Returning 3 Connections
Allocating 15 Connections
cp.curActConns = 33
Returning 4 Connections
cp.curActConns = 33
Returning 3 Connections
Allocating 8 Connections
cp.curActConns = 38
Returning 2 Connections
Allocating 3 Connections
cp.curActConns = 39
Returning 4 Connections
Allocating 9 Connections
cp.curActConns = 44
Returning 3 Connections
Allocating 7 Connections
cp.curActConns = 44
Returning 1 Connections
cp.curActConns = 47
Returning 4 Connections
Allocating 24 Connections
cp.curActConns = 55
Returning 0 Connections
cp.curActConns = 67
Returning 0 Connections
Allocating 13 Connections
cp.curActConns = 67
Returning 3 Connections
cp.curActConns = 77
Returning 3 Connections
Allocating 5 Connections
cp.curActConns = 78
Returning 1 Connections
cp.curActConns = 78
Allocating 10 Connections
Returning 0 Connections
cp.curActConns = 88
Returning 1 Connections
Allocating 16 Connections
cp.curActConns = 90
Returning 3 Connections
cp.curActConns = 100
Returning 1 Connections
Allocating 2 Connections
cp.curActConns = 101
Returning 3 Connections
Allocating 21 Connections
cp.curActConns = 101
Returning 3 Connections
cp.curActConns = 112
Returning 1 Connections
cp.curActConns = 115
Returning 2 Connections
Allocating 18 Connections
cp.curActConns = 124
Returning 2 Connections
cp.curActConns = 129
Returning 4 Connections
Allocating 2 Connections
cp.curActConns = 127
Returning 3 Connections
Allocating 21 Connections
cp.curActConns = 135
Returning 0 Connections
cp.curActConns = 145
Returning 3 Connections
Allocating 3 Connections
cp.curActConns = 145
Returning 2 Connections
Allocating 8 Connections
cp.curActConns = 147
Returning 1 Connections
cp.curActConns = 150
Returning 4 Connections
Allocating 8 Connections
cp.curActConns = 154
Returning 3 Connections
Allocating 16 Connections
cp.curActConns = 158
Returning 2 Connections
cp.curActConns = 165
Returning 3 Connections
Allocating 11 Connections
cp.curActConns = 169
Returning 1 Connections
cp.curActConns = 172
Returning 2 Connections
Allocating 15 Connections
cp.curActConns = 181
Returning 3 Connections
cp.curActConns = 182
Allocating 2 Connections
Returning 3 Connections
cp.curActConns = 181
Returning 0 Connections
Allocating 23 Connections
cp.curActConns = 192
Returning 0 Connections
cp.curActConns = 204
Returning 1 Connections
Allocating 0 Connections
cp.curActConns = 203
Returning 2 Connections
Allocating 2 Connections
cp.curActConns = 203
Returning 0 Connections
Allocating 0 Connections
cp.curActConns = 203
Returning 0 Connections
Allocating 0 Connections
cp.curActConns = 203
Returning 2 Connections
Allocating 3 Connections
cp.curActConns = 204
Returning 3 Connections
Allocating 1 Connections
cp.curActConns = 202
Returning 2 Connections
Allocating 4 Connections
cp.curActConns = 204
Returning 2 Connections
Allocating 2 Connections
cp.curActConns = 204
Returning 2 Connections
Allocating 1 Connections
cp.curActConns = 203
Returning 4 Connections
Allocating 1 Connections
cp.curActConns = 200
Returning 2 Connections
Allocating 6 Connections
cp.curActConns = 204
Returning 4 Connections
Allocating 1 Connections
cp.curActConns = 201
Returning 0 Connections
Allocating 3 Connections
cp.curActConns = 204
Returning 1 Connections
Allocating 4 Connections
cp.curActConns = 204
Returning 1 Connections
cp.curActConns = 206
Returning 2 Connections
Allocating 0 Connections
cp.curActConns = 204
Returning 0 Connections
Allocating 0 Connections
cp.curActConns = 204
Returning 2 Connections
Allocating 4 Connections
cp.curActConns = 206
Returning 3 Connections
Allocating 3 Connections
cp.curActConns = 206
Returning 4 Connections
Allocating 3 Connections
cp.curActConns = 205
Returning 4 Connections
Allocating 2 Connections
cp.curActConns = 203
Returning 2 Connections
Allocating 2 Connections
cp.curActConns = 203
Returning 1 Connections
Allocating 4 Connections
cp.curActConns = 206
Returning 0 Connections
Allocating 1 Connections
cp.curActConns = 207
Returning 1 Connections
Allocating 1 Connections
cp.curActConns = 207
Returning 0 Connections
Allocating 1 Connections
cp.curActConns = 208
Returning 4 Connections
Allocating 4 Connections
cp.curActConns = 208
Returning 0 Connections
Allocating 3 Connections
cp.curActConns = 211
Returning 1 Connections
Allocating 0 Connections
cp.curActConns = 210
Returning 2 Connections
Allocating 3 Connections
cp.curActConns = 211
Returning 2 Connections
Allocating 1 Connections
cp.curActConns = 210
Returning 3 Connections
Allocating 4 Connections
cp.curActConns = 211
Returning 2 Connections
Allocating 3 Connections
cp.curActConns = 212
Returning 2 Connections
Allocating 3 Connections
cp.curActConns = 213
Returning 3 Connections
Allocating 3 Connections
cp.curActConns = 213
Returning 1 Connections
Allocating 0 Connections
cp.curActConns = 212
Returning 0 Connections
Allocating 0 Connections
cp.curActConns = 212
Returning 4 Connections
Allocating 2 Connections
cp.curActConns = 210
Returning 2 Connections
Allocating 0 Connections
cp.curActConns = 208
Returning 1 Connections
Allocating 4 Connections
cp.curActConns = 211
Returning 0 Connections
Allocating 3 Connections
cp.curActConns = 214
Returning 0 Connections
Allocating 4 Connections
cp.curActConns = 215
Returning 0 Connections
cp.curActConns = 218
Returning 3 Connections
Allocating 0 Connections
cp.curActConns = 215
Returning 4 Connections
Allocating 2 Connections
cp.curActConns = 213
Returning 0 Connections
Allocating 0 Connections
cp.curActConns = 213
Returning 2 Connections
Allocating 4 Connections
cp.curActConns = 215
Returning 0 Connections
Allocating 3 Connections
cp.curActConns = 218
Returning 1 Connections
Allocating 0 Connections
cp.curActConns = 217
Returning 1 Connections
Allocating 0 Connections
cp.curActConns = 216
Returning 1 Connections
Allocating 4 Connections
cp.curActConns = 219
Returning 2 Connections
Allocating 1 Connections
cp.curActConns = 218
Returning 2 Connections
Allocating 1 Connections
cp.curActConns = 217
Returning 0 Connections
Allocating 4 Connections
cp.curActConns = 221
Returning 2 Connections
Allocating 2 Connections
cp.curActConns = 221
Retuning from startDeallocatorRoutine
Retuning from startAllocatorRoutine
2019-12-10T20:47:18.441+05:30 [Info] [Queryport-connpool:127.0.0.1:15151] ... stopped
2019-12-10T20:47:19.418+05:30 [Info] [Queryport-connpool:127.0.0.1:15151] Stopping releaseConnsRoutine
--- PASS: TestSustainedHighConns (59.04s)
=== RUN   TestLowWM
2019-12-10T20:47:22.442+05:30 [Info] [Queryport-connpool:127.0.0.1:15151] started poolsize 20 overflow 5 low WM 10 relConn batch size 2 ...
2019-12-10T20:48:22.458+05:30 [Info] [Queryport-connpool:127.0.0.1:15151] active conns 0, free conns 10
2019-12-10T20:49:22.474+05:30 [Info] [Queryport-connpool:127.0.0.1:15151] active conns 0, free conns 10
2019-12-10T20:49:27.952+05:30 [Info] [Queryport-connpool:127.0.0.1:15151] ... stopped
2019-12-10T20:49:28.475+05:30 [Info] [Queryport-connpool:127.0.0.1:15151] Stopping releaseConnsRoutine
--- PASS: TestLowWM (129.51s)
=== RUN   TestTotalConns
2019-12-10T20:49:31.953+05:30 [Info] [Queryport-connpool:127.0.0.1:15151] started poolsize 120 overflow 5 low WM 10 relConn batch size 10 ...
2019-12-10T20:49:46.011+05:30 [Info] [Queryport-connpool:127.0.0.1:15151] ... stopped
2019-12-10T20:49:46.958+05:30 [Info] [Queryport-connpool:127.0.0.1:15151] Stopping releaseConnsRoutine
--- PASS: TestTotalConns (18.06s)
=== RUN   TestUpdateTickRate
2019-12-10T20:49:50.012+05:30 [Info] [Queryport-connpool:127.0.0.1:15151] started poolsize 40 overflow 5 low WM 2 relConn batch size 2 ...
2019-12-10T20:50:10.830+05:30 [Info] [Queryport-connpool:127.0.0.1:15151] ... stopped
2019-12-10T20:50:11.018+05:30 [Info] [Queryport-connpool:127.0.0.1:15151] Stopping releaseConnsRoutine
--- PASS: TestUpdateTickRate (24.82s)
PASS
ok  	github.com/couchbase/indexing/secondary/queryport/client	322.703s
Starting server: attempt 1

Functional tests

2019/12/10 20:52:55 In init()
2019/12/10 20:52:55 DBG: ChangeIndexerSettings: nodes = [0xc00007cc80]
2019/12/10 20:52:55 DBG: ChangeIndexerSettings: adminurl = 127.0.0.1:9106 host 127.0.0.1 sport 9106
2019/12/10 20:52:55 Changing config key queryport.client.settings.backfillLimit to value 0
2019/12/10 20:52:55 DBG: ChangeIndexerSettings: nodes = [0xc0000b7e00]
2019/12/10 20:52:55 DBG: ChangeIndexerSettings: adminurl = 127.0.0.1:9106 host 127.0.0.1 sport 9106
2019/12/10 20:52:55 Changing config key indexer.api.enableTestServer to value true
2019/12/10 20:52:55 DBG: ChangeIndexerSettings: nodes = [0xc00007d000]
2019/12/10 20:52:55 DBG: ChangeIndexerSettings: adminurl = 127.0.0.1:9106 host 127.0.0.1 sport 9106
2019/12/10 20:52:55 Changing config key indexer.settings.persisted_snapshot_init_build.moi.interval to value 60000
2019/12/10 20:52:56 DBG: ChangeIndexerSettings: nodes = [0xc0000f64c0]
2019/12/10 20:52:56 DBG: ChangeIndexerSettings: adminurl = 127.0.0.1:9106 host 127.0.0.1 sport 9106
2019/12/10 20:52:56 Changing config key indexer.settings.persisted_snapshot.moi.interval to value 60000
2019/12/10 20:52:56 DBG: ChangeIndexerSettings: nodes = [0xc0000f65c0]
2019/12/10 20:52:56 DBG: ChangeIndexerSettings: adminurl = 127.0.0.1:9106 host 127.0.0.1 sport 9106
2019/12/10 20:52:56 Changing config key indexer.settings.log_level to value info
2019/12/10 20:52:56 DBG: ChangeIndexerSettings: nodes = [0xc000144d00]
2019/12/10 20:52:56 DBG: ChangeIndexerSettings: adminurl = 127.0.0.1:9106 host 127.0.0.1 sport 9106
2019/12/10 20:52:56 Changing config key indexer.settings.storage_mode.disable_upgrade to value true
2019/12/10 20:52:56 Using forestdb for creating indexes
2019/12/10 20:52:56 DBG: ChangeIndexerSettings: nodes = [0xc0000f66c0]
2019/12/10 20:52:56 DBG: ChangeIndexerSettings: adminurl = 127.0.0.1:9106 host 127.0.0.1 sport 9106
2019/12/10 20:52:56 Changing config key indexer.settings.storage_mode to value forestdb
2019-12-10T20:52:57.288+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:9106.  Error = EOF. Kill Pipe.
2019-12-10T20:52:57.288+05:30 [Error] WatcherServer.runOnce() : Watcher terminated unexpectedly.
2019/12/10 20:53:06 Data file exists. Skipping download
2019/12/10 20:53:06 Data file exists. Skipping download
2019/12/10 20:53:07 In DropAllSecondaryIndexes()
2019/12/10 20:53:07 Emptying the default bucket
2019/12/10 20:53:10 Flush Enabled on bucket default
2019/12/10 20:54:18 Flushed the bucket default
2019/12/10 20:54:23 Create Index On the empty default Bucket()
2019/12/10 20:54:26 Created the secondary index index_eyeColor. Waiting for it become active
2019/12/10 20:54:26 Index is now active
2019/12/10 20:54:26 Populating the default bucket
=== RUN   TestScanAfterBucketPopulate
2019/12/10 20:54:37 In TestScanAfterBucketPopulate()
2019/12/10 20:54:37 Create an index on empty bucket, populate the bucket and Run a scan on the index
2019/12/10 20:54:37 Using n1ql client
2019-12-10T20:54:37.899+05:30 [Info] creating GsiClient for 127.0.0.1:9000
2019-12-10T20:54:37.911+05:30 [Info] New settings received: 
{"indexer.api.enableTestServer":true,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfFname":"","indexer.settings.cpuProfile":false,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfFname":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":250,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.recovery.max_rollbacks":2,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":120000,"indexer.settings.send_buffer_size":1024,"indexer.settings.sliceBufSize":50000,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"forestdb","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.wal_size":4096,"projector.settings.log_level":"info","queryport.client.settings.backfillLimit":0,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100}
2019-12-10T20:54:37.964+05:30 [Info] MetadataProvider.SetClusterStatus(): healthy nodes 1 failed node 0 unhealthy node 0 add node 0
2019-12-10T20:54:37.964+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T20:54:37.964+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T20:54:37.964+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9106
2019-12-10T20:54:38.003+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9106.
2019-12-10T20:54:38.006+05:30 [Info] MetadataProvider: Updating indexer version to 4
2019-12-10T20:54:38.006+05:30 [Info] initialized currmeta 3 force true 
2019-12-10T20:54:38.006+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] started poolsize 5000 overflow 30 low WM 1000 relConn batch size 100 ...
2019-12-10T20:54:38.006+05:30 [Info] [GsiScanClient:"127.0.0.1:9107"] started ...
2019-12-10T20:54:38.007+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] open new connection ...
2019-12-10T20:54:38.008+05:30 [Info] GsiClient::UpdateUsecjson: using collatejson as data format between indexer and GsiClient
2019-12-10T20:54:38.008+05:30 [Info] GsiClient::UpdateUsecjson: using collatejson as data format between indexer and GsiClient
2019-12-10T20:54:38.008+05:30 [Info] GSIC[default/default-1575991477896699991] started ...
2019/12/10 20:54:38 Expected and Actual scan responses are the same
--- PASS: TestScanAfterBucketPopulate (0.34s)
=== RUN   TestRestartNilSnapshot
2019/12/10 20:54:38 In TestRestartNilSnapshot()
2019/12/10 20:54:43 Created the secondary index idx_age. Waiting for it become active
2019/12/10 20:54:43 Index is now active
2019/12/10 20:54:43 Restarting indexer process ...
2019/12/10 20:54:43 []
2019-12-10T20:54:43.447+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:9106.  Error = EOF. Kill Pipe.
2019-12-10T20:54:43.447+05:30 [Error] WatcherServer.runOnce() : Watcher terminated unexpectedly.
2019-12-10T20:54:43.447+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:9106.  Error = EOF. Kill Pipe.
2019-12-10T20:54:43.448+05:30 [Error] WatcherServer.runOnce() : Watcher terminated unexpectedly.
2019/12/10 20:54:53 Using n1ql client
2019-12-10T20:54:53.433+05:30 [Error] transport error between 127.0.0.1:37830->127.0.0.1:9107: write tcp 127.0.0.1:37830->127.0.0.1:9107: write: broken pipe
2019-12-10T20:54:53.433+05:30 [Error] [GsiScanClient:"127.0.0.1:9107"] 1624781479110971954 request transport failed `write tcp 127.0.0.1:37830->127.0.0.1:9107: write: broken pipe`
2019-12-10T20:54:53.433+05:30 [Error] PickRandom: Fail to find indexer for all index partitions. Num partition 1.  Partition with instances 0 
2019/12/10 20:54:53 Expected and Actual scan responses are the same
--- PASS: TestRestartNilSnapshot (15.36s)
=== RUN   TestThreeIndexCreates
2019/12/10 20:54:53 In TestThreeIndexCreates()
2019/12/10 20:54:58 Created the secondary index index_balance. Waiting for it become active
2019/12/10 20:54:58 Index is now active
2019/12/10 20:54:58 Create docs mutations
2019/12/10 20:54:58 Using n1ql client
2019/12/10 20:54:58 Expected and Actual scan responses are the same
2019/12/10 20:55:02 Created the secondary index index_email. Waiting for it become active
2019/12/10 20:55:02 Index is now active
2019/12/10 20:55:02 Create docs mutations
2019/12/10 20:55:03 Using n1ql client
2019/12/10 20:55:03 Expected and Actual scan responses are the same
2019/12/10 20:55:07 Created the secondary index index_pin. Waiting for it become active
2019/12/10 20:55:07 Index is now active
2019/12/10 20:55:07 Delete docs mutations
2019/12/10 20:55:08 Using n1ql client
2019/12/10 20:55:08 Expected and Actual scan responses are the same
--- PASS: TestThreeIndexCreates (14.80s)
=== RUN   TestMultipleIndexCreatesDropsWithMutations
2019/12/10 20:55:08 In TestThreeIndexCreates()
2019/12/10 20:55:12 Created the secondary index index_state. Waiting for it become active
2019/12/10 20:55:12 Index is now active
2019/12/10 20:55:12 Create docs mutations
2019/12/10 20:55:13 Using n1ql client
2019/12/10 20:55:13 Expected and Actual scan responses are the same
2019/12/10 20:55:18 Created the secondary index index_registered. Waiting for it become active
2019/12/10 20:55:18 Index is now active
2019/12/10 20:55:18 Create docs mutations
2019/12/10 20:55:18 Using n1ql client
2019/12/10 20:55:18 Expected and Actual scan responses are the same
2019/12/10 20:55:23 Created the secondary index index_gender. Waiting for it become active
2019/12/10 20:55:23 Index is now active
2019/12/10 20:55:23 Create docs mutations
2019/12/10 20:55:23 Using n1ql client
2019/12/10 20:55:24 Expected and Actual scan responses are the same
2019/12/10 20:55:24 Dropping the secondary index index_registered
2019/12/10 20:55:24 Index dropped
2019/12/10 20:55:24 Create docs mutations
2019/12/10 20:55:24 Delete docs mutations
2019/12/10 20:55:24 Using n1ql client
2019/12/10 20:55:25 Expected and Actual scan responses are the same
2019/12/10 20:55:30 Created the secondary index index_longitude. Waiting for it become active
2019/12/10 20:55:30 Index is now active
2019/12/10 20:55:30 Create docs mutations
2019/12/10 20:55:30 Using n1ql client
2019/12/10 20:55:30 Expected and Actual scan responses are the same
--- PASS: TestMultipleIndexCreatesDropsWithMutations (22.28s)
=== RUN   TestCreateDropScan
2019/12/10 20:55:30 In TestCreateDropScan()
2019/12/10 20:55:35 Created the secondary index index_cd. Waiting for it become active
2019/12/10 20:55:35 Index is now active
2019/12/10 20:55:35 Using n1ql client
2019/12/10 20:55:35 Expected and Actual scan responses are the same
2019/12/10 20:55:35 Dropping the secondary index index_cd
2019/12/10 20:55:35 Index dropped
2019/12/10 20:55:35 Using n1ql client
2019/12/10 20:55:35 Scan failed as expected with error: GSI index index_cd not found.
--- PASS: TestCreateDropScan (4.98s)
=== RUN   TestCreateDropCreate
2019/12/10 20:55:35 In TestCreateDropCreate()
2019/12/10 20:55:40 Created the secondary index index_cdc. Waiting for it become active
2019/12/10 20:55:40 Index is now active
2019/12/10 20:55:40 Using n1ql client
2019/12/10 20:55:40 Expected and Actual scan responses are the same
2019/12/10 20:55:40 Dropping the secondary index index_cdc
2019/12/10 20:55:40 Index dropped
2019/12/10 20:55:40 Using n1ql client
2019/12/10 20:55:40 Scan 2 failed as expected with error: GSI index index_cdc not found.
2019/12/10 20:55:45 Created the secondary index index_cdc. Waiting for it become active
2019/12/10 20:55:45 Index is now active
2019/12/10 20:55:45 Using n1ql client
2019/12/10 20:55:45 Expected and Actual scan responses are the same
2019/12/10 20:55:45 (Inclusion 1) Lengths of expected and actual scan results are 5044 and 5044. Num of docs in bucket = 10402
2019/12/10 20:55:45 Using n1ql client
2019/12/10 20:55:45 Expected and Actual scan responses are the same
2019/12/10 20:55:45 (Inclusion 3) Lengths of expected and actual scan results are 5044 and 5044. Num of docs in bucket = 10402
--- PASS: TestCreateDropCreate (9.87s)
=== RUN   TestCreate2Drop1Scan2
2019/12/10 20:55:45 In TestCreate2Drop1Scan2()
2019/12/10 20:55:50 Created the secondary index index_i1. Waiting for it become active
2019/12/10 20:55:50 Index is now active
2019/12/10 20:55:54 Created the secondary index index_i2. Waiting for it become active
2019/12/10 20:55:54 Index is now active
2019/12/10 20:55:54 Using n1ql client
2019/12/10 20:55:55 Expected and Actual scan responses are the same
2019/12/10 20:55:55 Using n1ql client
2019/12/10 20:55:55 Expected and Actual scan responses are the same
2019/12/10 20:55:55 Dropping the secondary index index_i1
2019/12/10 20:55:55 Index dropped
2019/12/10 20:55:55 Using n1ql client
2019/12/10 20:55:55 Expected and Actual scan responses are the same
--- PASS: TestCreate2Drop1Scan2 (10.14s)
=== RUN   TestIndexNameCaseSensitivity
2019/12/10 20:55:55 In TestIndexNameCaseSensitivity()
2019/12/10 20:56:00 Created the secondary index index_age. Waiting for it become active
2019/12/10 20:56:00 Index is now active
2019/12/10 20:56:00 Using n1ql client
2019/12/10 20:56:00 Expected and Actual scan responses are the same
2019/12/10 20:56:00 Using n1ql client
2019/12/10 20:56:00 Scan failed as expected with error: GSI index index_Age not found.
--- PASS: TestIndexNameCaseSensitivity (4.91s)
=== RUN   TestCreateDuplicateIndex
2019/12/10 20:56:00 In TestCreateDuplicateIndex()
2019/12/10 20:56:05 Created the secondary index index_di1. Waiting for it become active
2019/12/10 20:56:05 Index is now active
2019/12/10 20:56:05 Index found:  index_di1
2019/12/10 20:56:05 Create failed as expected with error: Index index_di1 already exists.
--- PASS: TestCreateDuplicateIndex (4.89s)
=== RUN   TestDropNonExistingIndex
2019/12/10 20:56:05 In TestDropNonExistingIndex()
2019/12/10 20:56:05 Dropping the secondary index 123456
2019/12/10 20:56:05 Index drop failed as expected with error: Index does not exist.
--- PASS: TestDropNonExistingIndex (0.00s)
=== RUN   TestCreateIndexNonExistentBucket
2019/12/10 20:56:05 In TestCreateIndexNonExistentBucket()
2019-12-10T20:56:05.910+05:30 [Error] Encountered error during create index.  Error: Bucket does not exist or temporarily unavailable for creating new index. Please retry the operation at a later time (err=Bucket not found).
2019-12-10T20:56:15.913+05:30 [Error] Fail to create index: Bucket does not exist or temporarily unavailable for creating new index. Please retry the operation at a later time (err=Bucket not found).
2019/12/10 20:56:15 Index create failed as expected with error: Bucket does not exist or temporarily unavailable for creating new index. Please retry the operation at a later time (err=Bucket not found).
--- PASS: TestCreateIndexNonExistentBucket (10.45s)
=== RUN   TestScanWithNoTimeout
2019/12/10 20:56:15 Create an index on empty bucket, populate the bucket and Run a scan on the index
2019/12/10 20:56:15 DBG: ChangeIndexerSettings: nodes = [0xc003acbf40]
2019/12/10 20:56:15 DBG: ChangeIndexerSettings: adminurl = 127.0.0.1:9106 host 127.0.0.1 sport 9106
2019/12/10 20:56:15 Changing config key indexer.settings.scan_timeout to value 0
2019/12/10 20:56:16 Using n1ql client
2019-12-10T20:56:16.113+05:30 [Info] GsiClient::UpdateUsecjson: using collatejson as data format between indexer and GsiClient
2019/12/10 20:56:16 Expected and Actual scan responses are the same
--- PASS: TestScanWithNoTimeout (0.60s)
=== RUN   TestIndexingOnBinaryBucketMeta
2019/12/10 20:56:16 In TestIndexingOnBinaryBucketMeta()
2019/12/10 20:56:16 	 1. Populate a bucekt with binary docs and create indexs on the `id`, `cas` and `expiration` fields of Metadata
2019/12/10 20:56:16 	 2. Validate the test by comparing the items_count of indexes and the number of docs in the bucket for each of the fields
2019/12/10 20:56:19 Modified parameters of bucket default
2019/12/10 20:56:19 Created bucket binaryBucket
2019/12/10 20:56:53 Created the secondary index index_binary_meta_id. Waiting for it become active
2019/12/10 20:56:53 Index is now active
2019/12/10 20:56:58 items_count stat is 10 for index index_binary_meta_id
2019/12/10 20:56:58 Dropping the secondary index index_binary_meta_id
2019/12/10 20:56:58 Index dropped
2019/12/10 20:57:02 Created the secondary index index_binary_meta_cas. Waiting for it become active
2019/12/10 20:57:02 Index is now active
2019/12/10 20:57:07 items_count stat is 10 for index index_binary_meta_cas
2019/12/10 20:57:07 Dropping the secondary index index_binary_meta_cas
2019/12/10 20:57:07 Index dropped
2019/12/10 20:57:11 Created the secondary index index_binary_meta_expiration. Waiting for it become active
2019/12/10 20:57:11 Index is now active
2019/12/10 20:57:16 items_count stat is 10 for index index_binary_meta_expiration
2019/12/10 20:57:16 Dropping the secondary index index_binary_meta_expiration
2019/12/10 20:57:16 Index dropped
2019/12/10 20:57:18 Deleted bucket binaryBucket
2019/12/10 20:57:21 Modified parameters of bucket default
--- PASS: TestIndexingOnBinaryBucketMeta (95.33s)
=== RUN   TestRetainDeleteXATTRBinaryDocs
2019/12/10 20:57:51 In TestRetainDeleteXATTRBinaryDocs()
2019/12/10 20:57:51 	 1. Populate a bucket with binary docs having system XATTRS
2019/12/10 20:57:51 	 2. Create index on the system XATTRS with "retain_deleted_xattr" attribute set to true
2019/12/10 20:57:51 	 3. Delete the documents in the bucket
2019/12/10 20:57:51 	 4. Query for the meta() information in the source bucket. The total number of results should be equivalent to the number of documents in the bucket before deletion of documents
2019/12/10 20:57:54 Modified parameters of bucket default
2019/12/10 20:57:54 Created bucket binaryBucket
2019/12/10 20:58:35 Created the secondary index index_system_xattr. Waiting for it become active
2019/12/10 20:58:35 Index is now active
2019/12/10 20:58:40 Deleted all the documents in the bucket: binaryBucket successfully
2019/12/10 20:58:42 Deleted bucket binaryBucket
2019/12/10 20:58:45 Modified parameters of bucket default
--- PASS: TestRetainDeleteXATTRBinaryDocs (83.34s)
=== RUN   TestSimpleIndex_FloatDataType
2019/12/10 20:59:15 In TestSimpleIndex_FloatDataType()
2019/12/10 20:59:15 Index found:  index_age
2019/12/10 20:59:15 Using n1ql client
2019/12/10 20:59:15 Expected and Actual scan responses are the same
--- PASS: TestSimpleIndex_FloatDataType (0.06s)
=== RUN   TestSimpleIndex_StringDataType
2019/12/10 20:59:15 In TestSimpleIndex_StringDataType()
2019/12/10 20:59:19 Created the secondary index index_company. Waiting for it become active
2019/12/10 20:59:19 Index is now active
2019/12/10 20:59:19 Using n1ql client
2019/12/10 20:59:20 Expected and Actual scan responses are the same
2019/12/10 20:59:20 Using n1ql client
2019/12/10 20:59:20 Expected and Actual scan responses are the same
--- PASS: TestSimpleIndex_StringDataType (5.00s)
=== RUN   TestSimpleIndex_FieldValueCaseSensitivity
2019/12/10 20:59:20 In TestSimpleIndex_StringCaseSensitivity()
2019/12/10 20:59:20 Index found:  index_company
2019/12/10 20:59:20 Using n1ql client
2019/12/10 20:59:20 Expected and Actual scan responses are the same
2019/12/10 20:59:20 Using n1ql client
2019/12/10 20:59:20 Expected and Actual scan responses are the same
--- PASS: TestSimpleIndex_FieldValueCaseSensitivity (0.22s)
=== RUN   TestSimpleIndex_BoolDataType
2019/12/10 20:59:20 In TestSimpleIndex_BoolDataType()
2019/12/10 20:59:25 Created the secondary index index_isActive. Waiting for it become active
2019/12/10 20:59:25 Index is now active
2019/12/10 20:59:25 Using n1ql client
2019/12/10 20:59:25 Expected and Actual scan responses are the same
--- PASS: TestSimpleIndex_BoolDataType (5.24s)
=== RUN   TestBasicLookup
2019/12/10 20:59:25 In TestBasicLookup()
2019/12/10 20:59:25 Index found:  index_company
2019/12/10 20:59:25 Using n1ql client
2019/12/10 20:59:25 Expected and Actual scan responses are the same
--- PASS: TestBasicLookup (0.01s)
=== RUN   TestIndexOnNonExistentField
2019/12/10 20:59:25 In TestIndexOnNonExistentField()
2019/12/10 20:59:30 Created the secondary index index_height. Waiting for it become active
2019/12/10 20:59:30 Index is now active
2019/12/10 20:59:30 Using n1ql client
2019/12/10 20:59:30 Expected and Actual scan responses are the same
--- PASS: TestIndexOnNonExistentField (4.90s)
=== RUN   TestIndexPartiallyMissingField
2019/12/10 20:59:30 In TestIndexPartiallyMissingField()
2019/12/10 20:59:35 Created the secondary index index_nationality. Waiting for it become active
2019/12/10 20:59:35 Index is now active
2019/12/10 20:59:35 Using n1ql client
2019/12/10 20:59:35 Expected and Actual scan responses are the same
--- PASS: TestIndexPartiallyMissingField (5.18s)
=== RUN   TestScanNonMatchingDatatype
2019/12/10 20:59:35 In TestScanNonMatchingDatatype()
2019/12/10 20:59:35 Index found:  index_age
2019/12/10 20:59:35 Using n1ql client
2019/12/10 20:59:35 Expected and Actual scan responses are the same
--- PASS: TestScanNonMatchingDatatype (0.02s)
=== RUN   TestInclusionNeither
2019/12/10 20:59:35 In TestInclusionNeither()
2019/12/10 20:59:35 Index found:  index_age
2019/12/10 20:59:35 Using n1ql client
2019/12/10 20:59:35 Expected and Actual scan responses are the same
--- PASS: TestInclusionNeither (0.12s)
=== RUN   TestInclusionLow
2019/12/10 20:59:35 In TestInclusionLow()
2019/12/10 20:59:35 Index found:  index_age
2019/12/10 20:59:35 Using n1ql client
2019/12/10 20:59:35 Expected and Actual scan responses are the same
--- PASS: TestInclusionLow (0.07s)
=== RUN   TestInclusionHigh
2019/12/10 20:59:35 In TestInclusionHigh()
2019/12/10 20:59:35 Index found:  index_age
2019/12/10 20:59:36 Using n1ql client
2019/12/10 20:59:36 Expected and Actual scan responses are the same
--- PASS: TestInclusionHigh (0.08s)
=== RUN   TestInclusionBoth
2019/12/10 20:59:36 In TestInclusionBoth()
2019/12/10 20:59:36 Index found:  index_age
2019/12/10 20:59:36 Using n1ql client
2019/12/10 20:59:36 Expected and Actual scan responses are the same
--- PASS: TestInclusionBoth (0.12s)
=== RUN   TestNestedIndex_String
2019/12/10 20:59:36 In TestNestedIndex_String()
2019/12/10 20:59:41 Created the secondary index index_streetname. Waiting for it become active
2019/12/10 20:59:41 Index is now active
2019/12/10 20:59:41 Using n1ql client
2019/12/10 20:59:41 Expected and Actual scan responses are the same
--- PASS: TestNestedIndex_String (4.91s)
=== RUN   TestNestedIndex_Float
2019/12/10 20:59:41 In TestNestedIndex_Float()
2019/12/10 20:59:46 Created the secondary index index_floor. Waiting for it become active
2019/12/10 20:59:46 Index is now active
2019/12/10 20:59:46 Using n1ql client
2019/12/10 20:59:46 Expected and Actual scan responses are the same
--- PASS: TestNestedIndex_Float (4.97s)
=== RUN   TestNestedIndex_Bool
2019/12/10 20:59:46 In TestNestedIndex_Bool()
2019/12/10 20:59:51 Created the secondary index index_isresidential. Waiting for it become active
2019/12/10 20:59:51 Index is now active
2019/12/10 20:59:51 Using n1ql client
2019/12/10 20:59:51 Expected and Actual scan responses are the same
--- PASS: TestNestedIndex_Bool (5.21s)
=== RUN   TestLookupJsonObject
2019/12/10 20:59:51 In TestLookupJsonObject()
2019/12/10 20:59:58 Created the secondary index index_streetaddress. Waiting for it become active
2019/12/10 20:59:58 Index is now active
2019/12/10 20:59:58 Using n1ql client
2019/12/10 20:59:58 Count of docScanResults is 1
2019/12/10 20:59:58 Key: User3bf51f08-0bac-4c03-bcec-5c255cbdde2c  Value: [map[doornumber:12B floor:5 buildingname:Sterling Heights streetname:Hill Street]]
2019/12/10 20:59:58 Count of scanResults is 1
2019/12/10 20:59:58 Key: string User3bf51f08-0bac-4c03-bcec-5c255cbdde2c  Value: value.Values [{"buildingname":"Sterling Heights","doornumber":"12B","floor":5,"streetname":"Hill Street"}] false
2019/12/10 20:59:58 Expected and Actual scan responses are the same
--- PASS: TestLookupJsonObject (6.80s)
=== RUN   TestLookupObjDifferentOrdering
2019/12/10 20:59:58 In TestLookupObjDifferentOrdering()
2019/12/10 20:59:58 Index found:  index_streetaddress
2019/12/10 20:59:58 Using n1ql client
2019/12/10 20:59:58 Count of docScanResults is 1
2019/12/10 20:59:58 Key: User3bf51f08-0bac-4c03-bcec-5c255cbdde2c  Value: [map[buildingname:Sterling Heights streetname:Hill Street doornumber:12B floor:5]]
2019/12/10 20:59:58 Count of scanResults is 1
2019/12/10 20:59:58 Key: string User3bf51f08-0bac-4c03-bcec-5c255cbdde2c  Value: value.Values [{"buildingname":"Sterling Heights","doornumber":"12B","floor":5,"streetname":"Hill Street"}] false
2019/12/10 20:59:58 Expected and Actual scan responses are the same
--- PASS: TestLookupObjDifferentOrdering (0.02s)
=== RUN   TestRangeJsonObject
2019/12/10 20:59:58 In TestRangeJsonObject()
2019/12/10 20:59:58 Index found:  index_streetaddress
2019/12/10 20:59:58 Using n1ql client
2019/12/10 20:59:58 Count of scanResults is 2
2019/12/10 20:59:58 Key: string User3bf51f08-0bac-4c03-bcec-5c255cbdde2c  Value: value.Values [{"buildingname":"Sterling Heights","doornumber":"12B","floor":5,"streetname":"Hill Street"}] false
2019/12/10 20:59:58 Key: string Userbb48952f-f8d1-4e04-a0e1-96b9019706fb  Value: value.Values [{"buildingname":"Rosewood Gardens","doornumber":"514","floor":2,"streetname":"Karweg Place"}] false
2019/12/10 20:59:58 Count of docScanResults is 2
2019/12/10 20:59:58 Key: User3bf51f08-0bac-4c03-bcec-5c255cbdde2c  Value: [map[floor:5 streetname:Hill Street buildingname:Sterling Heights doornumber:12B]]
2019/12/10 20:59:58 Key: Userbb48952f-f8d1-4e04-a0e1-96b9019706fb  Value: [map[streetname:Karweg Place buildingname:Rosewood Gardens doornumber:514 floor:2]]
2019/12/10 20:59:58 Expected and Actual scan responses are the same
--- PASS: TestRangeJsonObject (0.00s)
=== RUN   TestLookupFloatDiffForms
2019/12/10 20:59:58 In TestLookupFloatDiffForms()
2019/12/10 21:00:03 Created the secondary index index_latitude. Waiting for it become active
2019/12/10 21:00:03 Index is now active
2019/12/10 21:00:03 Scan 1
2019/12/10 21:00:03 Using n1ql client
2019/12/10 21:00:03 Expected and Actual scan responses are the same
2019/12/10 21:00:03 Scan 2
2019/12/10 21:00:03 Using n1ql client
2019/12/10 21:00:03 Expected and Actual scan responses are the same
2019/12/10 21:00:03 Scan 3
2019/12/10 21:00:03 Using n1ql client
2019/12/10 21:00:03 Expected and Actual scan responses are the same
2019/12/10 21:00:03 Scan 4
2019/12/10 21:00:03 Using n1ql client
2019/12/10 21:00:03 Expected and Actual scan responses are the same
2019/12/10 21:00:03 Scan 5
2019/12/10 21:00:03 Using n1ql client
2019/12/10 21:00:03 Expected and Actual scan responses are the same
2019/12/10 21:00:03 Scan 6
2019/12/10 21:00:03 Using n1ql client
2019/12/10 21:00:03 Expected and Actual scan responses are the same
--- PASS: TestLookupFloatDiffForms (5.48s)
=== RUN   TestRangeFloatInclVariations
2019/12/10 21:00:03 In TestRangeFloatInclVariations()
2019/12/10 21:00:03 Index found:  index_latitude
2019/12/10 21:00:03 Scan 1
2019/12/10 21:00:03 Using n1ql client
2019/12/10 21:00:03 Expected and Actual scan responses are the same
2019/12/10 21:00:03 Scan 2
2019/12/10 21:00:03 Using n1ql client
2019/12/10 21:00:03 Expected and Actual scan responses are the same
2019/12/10 21:00:03 Scan 3
2019/12/10 21:00:03 Using n1ql client
2019/12/10 21:00:03 Expected and Actual scan responses are the same
2019/12/10 21:00:03 Scan 4
2019/12/10 21:00:03 Using n1ql client
2019/12/10 21:00:03 Expected and Actual scan responses are the same
2019/12/10 21:00:03 Scan 5
2019/12/10 21:00:03 Using n1ql client
2019/12/10 21:00:03 Expected and Actual scan responses are the same
2019/12/10 21:00:03 Scan 6
2019/12/10 21:00:03 Using n1ql client
2019/12/10 21:00:03 Expected and Actual scan responses are the same
--- PASS: TestRangeFloatInclVariations (0.02s)
=== RUN   TestScanAll
2019/12/10 21:00:03 In TestScanAll()
2019/12/10 21:00:08 Created the secondary index index_name. Waiting for it become active
2019/12/10 21:00:08 Index is now active
2019/12/10 21:00:08 Length of docScanResults = 10502
2019/12/10 21:00:08 Using n1ql client
2019/12/10 21:00:08 Length of scanResults = 10502
2019/12/10 21:00:08 Expected and Actual scan responses are the same
--- PASS: TestScanAll (5.38s)
=== RUN   TestScanAllNestedField
2019/12/10 21:00:08 In TestScanAllNestedField()
2019/12/10 21:00:08 Index found:  index_streetname
2019/12/10 21:00:08 Length of docScanResults = 2
2019/12/10 21:00:08 Using n1ql client
2019/12/10 21:00:08 Length of scanResults = 2
2019/12/10 21:00:08 Expected and Actual scan responses are the same
--- PASS: TestScanAllNestedField (0.01s)
=== RUN   TestBasicPrimaryIndex
2019/12/10 21:00:08 In TestBasicPrimaryIndex()
2019/12/10 21:00:14 Created the secondary index index_p1. Waiting for it become active
2019/12/10 21:00:14 Index is now active
2019-12-10T21:00:14.834+05:30 [Error] transport error between 127.0.0.1:37680->127.0.0.1:9107: write tcp 127.0.0.1:37680->127.0.0.1:9107: write: broken pipe
2019-12-10T21:00:14.834+05:30 [Error] [GsiScanClient:"127.0.0.1:9107"]  request transport failed `write tcp 127.0.0.1:37680->127.0.0.1:9107: write: broken pipe`
2019-12-10T21:00:14.834+05:30 [Error] PickRandom: Fail to find indexer for all index partitions. Num partition 1.  Partition with instances 0 
2019/12/10 21:00:15 Expected and Actual scan responses are the same
2019/12/10 21:00:15 CountRange() expected and actual is:  1912 and 1912
2019/12/10 21:00:15 lookupkey for CountLookup() = Userb196c295-75b0-48f0-8b3f-365e3b67a42f
2019/12/10 21:00:15 CountLookup() = 1
--- PASS: TestBasicPrimaryIndex (6.21s)
=== RUN   TestBasicNullDataType
2019/12/10 21:00:15 In TestBasicNullDataType()
2019/12/10 21:00:15 Index found:  index_email
2019/12/10 21:00:15 Using n1ql client
2019/12/10 21:00:15 Expected and Actual scan responses are the same
--- PASS: TestBasicNullDataType (0.01s)
=== RUN   TestBasicArrayDataType_ScanAll
2019/12/10 21:00:15 In TestBasicArrayDataType_ScanAll()
2019/12/10 21:00:20 Created the secondary index index_tags. Waiting for it become active
2019/12/10 21:00:20 Index is now active
2019/12/10 21:00:20 Using n1ql client
2019/12/10 21:00:20 Expected and Actual scan responses are the same
--- PASS: TestBasicArrayDataType_ScanAll (5.67s)
=== RUN   TestBasicArrayDataType_Lookup
2019/12/10 21:00:20 In TestBasicArrayDataType_Lookup()
2019/12/10 21:00:22 Index found:  index_tags
2019/12/10 21:00:22 Count of scanResults is 1
2019/12/10 21:00:22 Key: string Usere46cea01-38f6-4e7b-92e5-69d64668ae75  Value: value.Values [["reprehenderit","tempor","officia","exercitation","labore","sunt","tempor"]] false
--- PASS: TestBasicArrayDataType_Lookup (2.00s)
=== RUN   TestArrayDataType_LookupMissingArrayValue
2019/12/10 21:00:22 In TestArrayDataType_LookupMissingArrayValue()
2019/12/10 21:00:22 Index found:  index_tags
2019/12/10 21:00:22 Count of scanResults is 0
--- PASS: TestArrayDataType_LookupMissingArrayValue (0.00s)
=== RUN   TestArrayDataType_LookupWrongOrder
2019/12/10 21:00:22 In TestArrayDataType_LookupWrongOrder()
2019/12/10 21:00:22 Index found:  index_tags
2019/12/10 21:00:22 Count of scanResults is 0
--- PASS: TestArrayDataType_LookupWrongOrder (0.00s)
=== RUN   TestArrayDataType_LookupSubset
2019/12/10 21:00:22 In TestArrayDataType_LookupSubset()
2019/12/10 21:00:22 Index found:  index_tags
2019/12/10 21:00:22 Count of scanResults is 0
--- PASS: TestArrayDataType_LookupSubset (0.00s)
=== RUN   TestScanLimitParameter
2019/12/10 21:00:22 In TestScanLimitParameter()
2019/12/10 21:00:22 Index found:  index_age
2019/12/10 21:00:22 Using n1ql client
2019/12/10 21:00:22 Using n1ql client
--- PASS: TestScanLimitParameter (0.03s)
=== RUN   TestCountRange
2019/12/10 21:00:22 In TestRangeCount()
2019/12/10 21:00:22 Index found:  index_age
2019/12/10 21:00:22 Count of expected and actual Range is:  2384 and 2384
2019/12/10 21:00:23 Count of expected and actual Range is: 10002 and 10002
2019/12/10 21:00:23 Count of expected and actual Range are: 0 and 0
2019/12/10 21:00:23 Count of expected and actual Range are: 489 and 489
2019/12/10 21:00:23 Testing CountRange() for key <= val
2019/12/10 21:00:23 Count of expected and actual CountRange for key <= 30 are: 5231 and 5231
2019/12/10 21:00:23 Testing CountRange() for key >= val
2019/12/10 21:00:23 Count of expected and actual CountRange for key >= 25 are: 7688 and 7688
2019/12/10 21:00:23 Testing CountRange() for null < key <= val
2019/12/10 21:00:23 Count of expected and actual CountRange for key > null && key <= 30 are: 5231 and 5231
2019/12/10 21:00:23 Testing CountRange() for val <= key < null 
2019/12/10 21:00:23 Count of expected and actual CountRange for key >= 25 && key < null are: 0 and 0
2019/12/10 21:00:23 Count of expected and actual Range are: 0 and 0
--- PASS: TestCountRange (0.56s)
=== RUN   TestCountLookup
2019/12/10 21:00:23 In TestCountLookup()
2019/12/10 21:00:23 Index found:  index_age
2019/12/10 21:00:23 Count of expected and actual Range are: 501 and 501
2019/12/10 21:00:23 Count of expected and actual Range are: 0 and 0
--- PASS: TestCountLookup (0.02s)
=== RUN   TestRangeStatistics
2019/12/10 21:00:23 In TestRangeCount()
2019/12/10 21:00:23 Index found:  index_age
--- PASS: TestRangeStatistics (0.00s)
=== RUN   TestIndexCreateWithWhere
2019/12/10 21:00:23 In TestIndexCreateWithWhere()
2019/12/10 21:00:28 Created the secondary index index_ageabove30. Waiting for it become active
2019/12/10 21:00:28 Index is now active
2019/12/10 21:00:28 Using n1ql client
2019/12/10 21:00:28 Expected and Actual scan responses are the same
2019/12/10 21:00:28 Lengths of expected and actual scanReuslts are:  4282 and 4282
2019/12/10 21:00:33 Created the secondary index index_ageteens. Waiting for it become active
2019/12/10 21:00:33 Index is now active
2019/12/10 21:00:33 Using n1ql client
2019/12/10 21:00:33 Expected and Actual scan responses are the same
2019/12/10 21:00:33 Lengths of expected and actual scanReuslts are:  0 and 0
2019/12/10 21:00:38 Created the secondary index index_age35to45. Waiting for it become active
2019/12/10 21:00:38 Index is now active
2019/12/10 21:00:38 Using n1ql client
2019/12/10 21:00:38 Expected and Actual scan responses are the same
2019/12/10 21:00:38 Lengths of expected and actual scanReuslts are:  2873 and 2873
--- PASS: TestIndexCreateWithWhere (15.24s)
=== RUN   TestDeferredIndexCreate
2019/12/10 21:00:38 In TestDeferredIndexCreate()
2019/12/10 21:00:39 Created the secondary index index_deferred
2019/12/10 21:00:39 Created the index index_deferred in deferred mode. Index state is INDEX_STATE_READY
2019/12/10 21:00:41 Build the deferred index index_deferred. Waiting for the index to become active
2019/12/10 21:00:41 Waiting for index to go active ...
2019/12/10 21:00:42 Waiting for index to go active ...
2019/12/10 21:00:43 Waiting for index to go active ...
2019/12/10 21:00:44 Index is now active
2019/12/10 21:00:44 Using n1ql client
2019/12/10 21:00:44 Expected and Actual scan responses are the same
--- PASS: TestDeferredIndexCreate (6.04s)
=== RUN   TestCompositeIndex_NumAndString
2019/12/10 21:00:44 In TestCompositeIndex()
2019/12/10 21:00:49 Created the secondary index index_composite1. Waiting for it become active
2019/12/10 21:00:49 Index is now active
2019/12/10 21:00:49 Using n1ql client
2019/12/10 21:00:50 Using n1ql client
2019/12/10 21:00:50 Using n1ql client
2019/12/10 21:00:50 Expected and Actual scan responses are the same
--- PASS: TestCompositeIndex_NumAndString (5.67s)
=== RUN   TestCompositeIndex_TwoNumberFields
2019/12/10 21:00:50 In TestCompositeIndex()
2019/12/10 21:00:55 Created the secondary index index_composite2. Waiting for it become active
2019/12/10 21:00:55 Index is now active
2019/12/10 21:00:55 Using n1ql client
--- PASS: TestCompositeIndex_TwoNumberFields (5.72s)
=== RUN   TestNumbers_Int64_Float64
2019/12/10 21:00:56 In TestNumbers_Int64_Float64()
2019/12/10 21:01:01 Created the secondary index idx_numbertest. Waiting for it become active
2019/12/10 21:01:01 Index is now active
2019/12/10 21:01:01 
 ==== Int64 test #0
2019/12/10 21:01:01 Using n1ql client
2019/12/10 21:01:01 Expected and Actual scan responses are the same
2019/12/10 21:01:01 
 ==== Int64 test #1
2019/12/10 21:01:01 Using n1ql client
2019/12/10 21:01:01 Expected and Actual scan responses are the same
2019/12/10 21:01:01 
 ==== Int64 test #2
2019/12/10 21:01:01 Using n1ql client
2019/12/10 21:01:02 Expected and Actual scan responses are the same
2019/12/10 21:01:02 
 ==== Int64 test #3
2019/12/10 21:01:02 Using n1ql client
2019/12/10 21:01:02 Expected and Actual scan responses are the same
2019/12/10 21:01:02 
 ==== Int64 test #4
2019/12/10 21:01:02 Using n1ql client
2019/12/10 21:01:02 Expected and Actual scan responses are the same
2019/12/10 21:01:02 
 ==== Int64 test #5
2019/12/10 21:01:02 Using n1ql client
2019/12/10 21:01:02 Expected and Actual scan responses are the same
2019/12/10 21:01:02 
 ==== Int64 test #6
2019/12/10 21:01:02 Using n1ql client
2019/12/10 21:01:02 Expected and Actual scan responses are the same
2019/12/10 21:01:02 
 ==== Int64 test #7
2019/12/10 21:01:02 Using n1ql client
2019/12/10 21:01:03 Expected and Actual scan responses are the same
2019/12/10 21:01:03 
 ==== Int64 test #8
2019/12/10 21:01:03 Using n1ql client
2019/12/10 21:01:03 Expected and Actual scan responses are the same
2019/12/10 21:01:03 
 ==== Float64 test #0
2019/12/10 21:01:03 Using n1ql client
2019/12/10 21:01:03 Expected and Actual scan responses are the same
2019/12/10 21:01:03 
 ==== Float64 test #1
2019/12/10 21:01:03 Using n1ql client
2019/12/10 21:01:03 Expected and Actual scan responses are the same
2019/12/10 21:01:03 
 ==== Float64 test #2
2019/12/10 21:01:03 Using n1ql client
2019/12/10 21:01:03 Expected and Actual scan responses are the same
2019/12/10 21:01:03 
 ==== Float64 test #3
2019/12/10 21:01:03 Using n1ql client
2019/12/10 21:01:04 Expected and Actual scan responses are the same
--- PASS: TestNumbers_Int64_Float64 (7.88s)
=== RUN   TestRestartIndexer
2019/12/10 21:01:04 In TestRestartIndexer()
2019/12/10 21:01:04 []
2019-12-10T21:01:04.084+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:9106.  Error = EOF. Kill Pipe.
2019-12-10T21:01:04.084+05:30 [Error] WatcherServer.runOnce() : Watcher terminated unexpectedly.
2019-12-10T21:01:04.085+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:9106.  Error = EOF. Kill Pipe.
2019-12-10T21:01:04.085+05:30 [Error] WatcherServer.runOnce() : Watcher terminated unexpectedly.
2019/12/10 21:01:34 Using n1ql client
2019-12-10T21:01:34.055+05:30 [Error] transport error between 127.0.0.1:37958->127.0.0.1:9107: write tcp 127.0.0.1:37958->127.0.0.1:9107: write: broken pipe
2019-12-10T21:01:34.055+05:30 [Error] [GsiScanClient:"127.0.0.1:9107"] 7785351337262775217 request transport failed `write tcp 127.0.0.1:37958->127.0.0.1:9107: write: broken pipe`
2019/12/10 21:01:34 Len of expected and actual scan results are :  10002 and 10002
2019/12/10 21:01:34 Expected and Actual scan responses are the same
--- PASS: TestRestartIndexer (30.23s)
=== RUN   TestCreateDocsMutation
2019/12/10 21:01:34 In TestCreateDocsMutation()
2019/12/10 21:01:34 Index found:  index_age
2019/12/10 21:01:34 Using n1ql client
2019/12/10 21:01:34 Len of expected and actual scan results are :  10002 and 10002
2019/12/10 21:01:34 Expected and Actual scan responses are the same
2019/12/10 21:01:34 Using n1ql client
2019/12/10 21:01:35 Index Scan after mutations took 611.063033ms
2019/12/10 21:01:35 Len of expected and actual scan results are :  10102 and 10102
2019/12/10 21:01:35 Expected and Actual scan responses are the same
--- PASS: TestCreateDocsMutation (1.05s)
=== RUN   TestRestartProjector
2019/12/10 21:01:35 In TestRestartProjector()
2019/12/10 21:01:35 []
2019/12/10 21:02:05 Using n1ql client
2019/12/10 21:02:05 Len of expected and actual scan results are :  10102 and 10102
2019/12/10 21:02:05 Expected and Actual scan responses are the same
--- PASS: TestRestartProjector (30.22s)
=== RUN   TestDeleteDocsMutation
2019/12/10 21:02:05 In TestDeleteDocsMutation()
2019/12/10 21:02:05 Index found:  index_age
2019/12/10 21:02:05 Using n1ql client
2019/12/10 21:02:05 Len of expected and actual scan results are :  10102 and 10102
2019/12/10 21:02:05 Expected and Actual scan responses are the same
2019/12/10 21:02:05 Using n1ql client
2019/12/10 21:02:06 Index Scan after mutations took 778.178199ms
2019/12/10 21:02:06 Len of expected and actual scan results are :  9902 and 9902
2019/12/10 21:02:06 Expected and Actual scan responses are the same
--- PASS: TestDeleteDocsMutation (1.17s)
=== RUN   TestUpdateDocsMutation
2019/12/10 21:02:06 In TestUpdateDocsMutation()
2019/12/10 21:02:06 Index found:  index_age
2019/12/10 21:02:06 Using n1ql client
2019/12/10 21:02:06 Len of expected and actual scan results are :  9424 and 9424
2019/12/10 21:02:06 Expected and Actual scan responses are the same
2019/12/10 21:02:06 Num of keysFromMutDocs: 100
2019/12/10 21:02:06 Updating number of documents: 100
2019/12/10 21:02:07 Using n1ql client
2019/12/10 21:02:07 Index Scan after mutations took 397.823491ms
2019/12/10 21:02:07 Len of expected and actual scan results are :  9429 and 9429
2019/12/10 21:02:07 Expected and Actual scan responses are the same
--- PASS: TestUpdateDocsMutation (0.76s)
=== RUN   TestLargeMutations
2019/12/10 21:02:07 In TestLargeMutations()
2019/12/10 21:02:07 In DropAllSecondaryIndexes()
2019/12/10 21:02:07 Index found:  index_p1
2019/12/10 21:02:07 Dropped index index_p1
2019/12/10 21:02:07 Index found:  index_eyeColor
2019/12/10 21:02:07 Dropped index index_eyeColor
2019/12/10 21:02:07 Index found:  index_tags
2019/12/10 21:02:07 Dropped index index_tags
2019/12/10 21:02:07 Index found:  index_di1
2019/12/10 21:02:08 Dropped index index_di1
2019/12/10 21:02:08 Index found:  index_deferred
2019/12/10 21:02:08 Dropped index index_deferred
2019/12/10 21:02:08 Index found:  index_ageabove30
2019/12/10 21:02:08 Dropped index index_ageabove30
2019/12/10 21:02:08 Index found:  index_cdc
2019/12/10 21:02:08 Dropped index index_cdc
2019/12/10 21:02:08 Index found:  index_longitude
2019/12/10 21:02:08 Dropped index index_longitude
2019/12/10 21:02:08 Index found:  index_composite1
2019/12/10 21:02:08 Dropped index index_composite1
2019/12/10 21:02:08 Index found:  index_i2
2019/12/10 21:02:08 Dropped index index_i2
2019/12/10 21:02:08 Index found:  index_height
2019/12/10 21:02:09 Dropped index index_height
2019/12/10 21:02:09 Index found:  index_composite2
2019/12/10 21:02:09 Dropped index index_composite2
2019/12/10 21:02:09 Index found:  index_isresidential
2019/12/10 21:02:09 Dropped index index_isresidential
2019/12/10 21:02:09 Index found:  idx_numbertest
2019/12/10 21:02:09 Dropped index idx_numbertest
2019/12/10 21:02:09 Index found:  index_streetaddress
2019/12/10 21:02:09 Dropped index index_streetaddress
2019/12/10 21:02:09 Index found:  index_email
2019/12/10 21:02:09 Dropped index index_email
2019/12/10 21:02:09 Index found:  idx_age
2019/12/10 21:02:10 Dropped index idx_age
2019/12/10 21:02:10 Index found:  index_nationality
2019/12/10 21:02:10 Dropped index index_nationality
2019/12/10 21:02:10 Index found:  index_pin
2019/12/10 21:02:10 Dropped index index_pin
2019/12/10 21:02:10 Index found:  index_age
2019/12/10 21:02:10 Dropped index index_age
2019/12/10 21:02:10 Index found:  index_floor
2019/12/10 21:02:10 Dropped index index_floor
2019/12/10 21:02:10 Index found:  index_company
2019/12/10 21:02:10 Dropped index index_company
2019/12/10 21:02:10 Index found:  index_ageteens
2019/12/10 21:02:11 Dropped index index_ageteens
2019/12/10 21:02:11 Index found:  index_state
2019/12/10 21:02:11 Dropped index index_state
2019/12/10 21:02:11 Index found:  index_gender
2019/12/10 21:02:11 Dropped index index_gender
2019/12/10 21:02:11 Index found:  index_isActive
2019/12/10 21:02:11 Dropped index index_isActive
2019/12/10 21:02:11 Index found:  index_latitude
2019/12/10 21:02:11 Dropped index index_latitude
2019/12/10 21:02:11 Index found:  index_name
2019/12/10 21:02:11 Dropped index index_name
2019/12/10 21:02:11 Index found:  index_balance
2019/12/10 21:02:12 Dropped index index_balance
2019/12/10 21:02:12 Index found:  index_streetname
2019/12/10 21:02:12 Dropped index index_streetname
2019/12/10 21:02:12 Index found:  index_age35to45
2019/12/10 21:02:12 Dropped index index_age35to45
2019/12/10 21:02:35 Created the secondary index indexmut_1. Waiting for it become active
2019/12/10 21:02:35 Index is now active
2019/12/10 21:02:35 Using n1ql client
2019/12/10 21:02:36 Expected and Actual scan responses are the same
2019/12/10 21:02:36 Len of expected and actual scan results are :  29902 and 29902
2019/12/10 21:02:36 ITERATION 0
2019/12/10 21:02:56 Created the secondary index indexmut_2. Waiting for it become active
2019/12/10 21:02:56 Index is now active
2019/12/10 21:02:56 Using n1ql client
2019/12/10 21:02:57 Expected and Actual scan responses are the same
2019/12/10 21:02:57 Len of expected and actual scan results are :  39902 and 39902
2019/12/10 21:02:57 Using n1ql client
2019/12/10 21:02:57 Expected and Actual scan responses are the same
2019/12/10 21:02:57 Len of expected and actual scan results are :  39902 and 39902
2019/12/10 21:02:57 Dropping the secondary index indexmut_2
2019/12/10 21:02:58 Index dropped
2019/12/10 21:02:58 ITERATION 1
2019/12/10 21:03:17 Created the secondary index indexmut_2. Waiting for it become active
2019/12/10 21:03:17 Index is now active
2019/12/10 21:03:17 Using n1ql client
2019/12/10 21:03:18 Expected and Actual scan responses are the same
2019/12/10 21:03:18 Len of expected and actual scan results are :  49902 and 49902
2019/12/10 21:03:18 Using n1ql client
2019/12/10 21:03:20 Expected and Actual scan responses are the same
2019/12/10 21:03:20 Len of expected and actual scan results are :  49902 and 49902
2019/12/10 21:03:20 Dropping the secondary index indexmut_2
2019/12/10 21:03:20 Index dropped
2019/12/10 21:03:20 ITERATION 2
2019/12/10 21:03:40 Created the secondary index indexmut_2. Waiting for it become active
2019/12/10 21:03:40 Index is now active
2019/12/10 21:03:41 Using n1ql client
2019/12/10 21:03:42 Expected and Actual scan responses are the same
2019/12/10 21:03:42 Len of expected and actual scan results are :  59902 and 59902
2019/12/10 21:03:42 Using n1ql client
2019/12/10 21:03:43 Expected and Actual scan responses are the same
2019/12/10 21:03:43 Len of expected and actual scan results are :  59902 and 59902
2019/12/10 21:03:43 Dropping the secondary index indexmut_2
2019/12/10 21:03:43 Index dropped
2019/12/10 21:03:43 ITERATION 3
2019/12/10 21:04:06 Created the secondary index indexmut_2. Waiting for it become active
2019/12/10 21:04:06 Index is now active
2019/12/10 21:04:06 Using n1ql client
2019/12/10 21:04:07 Expected and Actual scan responses are the same
2019/12/10 21:04:07 Len of expected and actual scan results are :  69902 and 69902
2019/12/10 21:04:08 Using n1ql client
2019/12/10 21:04:09 Expected and Actual scan responses are the same
2019/12/10 21:04:09 Len of expected and actual scan results are :  69902 and 69902
2019/12/10 21:04:09 Dropping the secondary index indexmut_2
2019/12/10 21:04:09 Index dropped
2019/12/10 21:04:09 ITERATION 4
2019/12/10 21:04:33 Created the secondary index indexmut_2. Waiting for it become active
2019/12/10 21:04:33 Index is now active
2019/12/10 21:04:34 Using n1ql client
2019/12/10 21:04:36 Expected and Actual scan responses are the same
2019/12/10 21:04:36 Len of expected and actual scan results are :  79902 and 79902
2019/12/10 21:04:36 Using n1ql client
2019/12/10 21:04:38 Expected and Actual scan responses are the same
2019/12/10 21:04:38 Len of expected and actual scan results are :  79902 and 79902
2019/12/10 21:04:38 Dropping the secondary index indexmut_2
2019/12/10 21:04:38 Index dropped
2019/12/10 21:04:38 ITERATION 5
2019/12/10 21:05:03 Created the secondary index indexmut_2. Waiting for it become active
2019/12/10 21:05:03 Index is now active
2019/12/10 21:05:03 Using n1ql client
2019/12/10 21:05:05 Expected and Actual scan responses are the same
2019/12/10 21:05:05 Len of expected and actual scan results are :  89902 and 89902
2019/12/10 21:05:05 Using n1ql client
2019/12/10 21:05:07 Expected and Actual scan responses are the same
2019/12/10 21:05:07 Len of expected and actual scan results are :  89902 and 89902
2019/12/10 21:05:07 Dropping the secondary index indexmut_2
2019/12/10 21:05:07 Index dropped
2019/12/10 21:05:07 ITERATION 6
2019/12/10 21:05:32 Created the secondary index indexmut_2. Waiting for it become active
2019/12/10 21:05:32 Index is now active
2019/12/10 21:05:32 Using n1ql client
2019/12/10 21:05:34 Expected and Actual scan responses are the same
2019/12/10 21:05:34 Len of expected and actual scan results are :  99902 and 99902
2019/12/10 21:05:34 Using n1ql client
2019/12/10 21:05:36 Expected and Actual scan responses are the same
2019/12/10 21:05:36 Len of expected and actual scan results are :  99902 and 99902
2019/12/10 21:05:36 Dropping the secondary index indexmut_2
2019/12/10 21:05:36 Index dropped
2019/12/10 21:05:36 ITERATION 7
2019/12/10 21:06:03 Created the secondary index indexmut_2. Waiting for it become active
2019/12/10 21:06:03 Index is now active
2019/12/10 21:06:03 Using n1ql client
2019/12/10 21:06:05 Expected and Actual scan responses are the same
2019/12/10 21:06:05 Len of expected and actual scan results are :  109902 and 109902
2019/12/10 21:06:06 Using n1ql client
2019/12/10 21:06:08 Expected and Actual scan responses are the same
2019/12/10 21:06:08 Len of expected and actual scan results are :  109902 and 109902
2019/12/10 21:06:08 Dropping the secondary index indexmut_2
2019/12/10 21:06:08 Index dropped
2019/12/10 21:06:08 ITERATION 8
2019/12/10 21:06:37 Created the secondary index indexmut_2. Waiting for it become active
2019/12/10 21:06:37 Index is now active
2019/12/10 21:06:37 Using n1ql client
2019/12/10 21:06:40 Expected and Actual scan responses are the same
2019/12/10 21:06:40 Len of expected and actual scan results are :  119902 and 119902
2019/12/10 21:06:40 Using n1ql client
2019/12/10 21:06:43 Expected and Actual scan responses are the same
2019/12/10 21:06:43 Len of expected and actual scan results are :  119902 and 119902
2019/12/10 21:06:43 Dropping the secondary index indexmut_2
2019/12/10 21:06:44 Index dropped
2019/12/10 21:06:44 ITERATION 9
2019/12/10 21:07:14 Created the secondary index indexmut_2. Waiting for it become active
2019/12/10 21:07:14 Index is now active
2019/12/10 21:07:14 Using n1ql client
2019/12/10 21:07:16 Expected and Actual scan responses are the same
2019/12/10 21:07:16 Len of expected and actual scan results are :  129902 and 129902
2019/12/10 21:07:16 Using n1ql client
2019/12/10 21:07:19 Expected and Actual scan responses are the same
2019/12/10 21:07:19 Len of expected and actual scan results are :  129902 and 129902
2019/12/10 21:07:19 Dropping the secondary index indexmut_2
2019/12/10 21:07:20 Index dropped
2019/12/10 21:07:20 ITERATION 10
2019/12/10 21:07:48 Created the secondary index indexmut_2. Waiting for it become active
2019/12/10 21:07:48 Index is now active
2019/12/10 21:07:49 Using n1ql client
2019/12/10 21:07:52 Expected and Actual scan responses are the same
2019/12/10 21:07:52 Len of expected and actual scan results are :  139902 and 139902
2019/12/10 21:07:52 Using n1ql client
2019/12/10 21:07:56 Expected and Actual scan responses are the same
2019/12/10 21:07:56 Len of expected and actual scan results are :  139902 and 139902
2019/12/10 21:07:56 Dropping the secondary index indexmut_2
2019/12/10 21:07:56 Index dropped
--- PASS: TestLargeMutations (348.69s)
=== RUN   TestPlanner
2019/12/10 21:07:56 In TestPlanner()
2019/12/10 21:07:56 -------------------------------------------
2019/12/10 21:07:56 initial placement - 20-50M, 10 index, 3 replica, 2x
2019-12-10T21:07:56.190+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T21:07:56.190+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T21:07:56.194+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T21:07:56.194+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T21:07:56.196+05:30 [Info] switched currmeta from 449 -> 451 force true 
2019-12-10T21:07:56.200+05:30 [Info] switched currmeta from 455 -> 455 force true 
2019-12-10T21:07:56.236+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] active conns 0, free conns 1
2019-12-10T21:07:56.285+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T21:07:56.285+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T21:07:56.287+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T21:07:56.287+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T21:07:56.293+05:30 [Info] switched currmeta from 455 -> 455 force true 
2019-12-10T21:07:56.293+05:30 [Info] switched currmeta from 451 -> 451 force true 
2019-12-10T21:07:56.491+05:30 [Info] Score: 0.07691315920108818
2019-12-10T21:07:56.491+05:30 [Info] Memory Quota: 56480192476 (52.6013G)
2019-12-10T21:07:56.491+05:30 [Info] CPU Quota: 12
2019-12-10T21:07:56.491+05:30 [Info] Indexer Memory Mean 33948867794 (31.6173G)
2019-12-10T21:07:56.491+05:30 [Info] Indexer Memory Deviation 2611114673 (2.43179G) (7.69%)
2019-12-10T21:07:56.491+05:30 [Info] Indexer Memory Utilization 0.6011
2019-12-10T21:07:56.491+05:30 [Info] Indexer CPU Mean 8.6968
2019-12-10T21:07:56.491+05:30 [Info] Indexer CPU Deviation 2.93 (33.70%)
2019-12-10T21:07:56.491+05:30 [Info] Indexer CPU Utilization 0.7247
2019-12-10T21:07:56.491+05:30 [Info] Indexer IO Mean 0.0000
2019-12-10T21:07:56.491+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2019-12-10T21:07:56.491+05:30 [Info] Indexer Drain Rate Mean 0.0000
2019-12-10T21:07:56.491+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2019-12-10T21:07:56.491+05:30 [Info] Indexer Scan Rate Mean 0.0000
2019-12-10T21:07:56.491+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2019-12-10T21:07:56.491+05:30 [Info] Indexer Data Size Mean 33948867794 (31.6173G)
2019-12-10T21:07:56.491+05:30 [Info] Indexer Data Size Deviation 2611114673 (2.43179G) (7.69%)
2019-12-10T21:07:56.491+05:30 [Info] Total Index Data (from non-deleted node) 0
2019-12-10T21:07:56.491+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2019-12-10T21:07:56.491+05:30 [Info] No. Index (from non-deleted node) 0
2019-12-10T21:07:56.491+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2019/12/10 21:07:56 -------------------------------------------
2019/12/10 21:07:56 initial placement - 20-50M, 30 index, 3 replica, 2x
2019-12-10T21:07:57.556+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T21:07:57.617+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T21:07:57.617+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T21:07:57.621+05:30 [Info] switched currmeta from 455 -> 455 force true 
2019-12-10T21:07:57.623+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T21:07:57.623+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T21:07:57.626+05:30 [Info] switched currmeta from 451 -> 451 force true 
2019-12-10T21:07:58.530+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T21:07:58.559+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T21:07:58.559+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T21:07:58.564+05:30 [Info] switched currmeta from 451 -> 451 force true 
2019-12-10T21:07:58.569+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T21:07:58.569+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T21:07:58.571+05:30 [Info] switched currmeta from 455 -> 455 force true 
2019-12-10T21:07:59.724+05:30 [Info] Score: 0.06310749869037471
2019-12-10T21:07:59.724+05:30 [Info] Memory Quota: 56361968784 (52.4912G)
2019-12-10T21:07:59.724+05:30 [Info] CPU Quota: 12
2019-12-10T21:07:59.724+05:30 [Info] Indexer Memory Mean 35264996985 (32.8431G)
2019-12-10T21:07:59.724+05:30 [Info] Indexer Memory Deviation 2225485751 (2.07265G) (6.31%)
2019-12-10T21:07:59.724+05:30 [Info] Indexer Memory Utilization 0.6257
2019-12-10T21:07:59.724+05:30 [Info] Indexer CPU Mean 10.0316
2019-12-10T21:07:59.724+05:30 [Info] Indexer CPU Deviation 2.28 (22.74%)
2019-12-10T21:07:59.724+05:30 [Info] Indexer CPU Utilization 0.8360
2019-12-10T21:07:59.724+05:30 [Info] Indexer IO Mean 0.0000
2019-12-10T21:07:59.724+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2019-12-10T21:07:59.724+05:30 [Info] Indexer Drain Rate Mean 0.0000
2019-12-10T21:07:59.724+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2019-12-10T21:07:59.724+05:30 [Info] Indexer Scan Rate Mean 0.0000
2019-12-10T21:07:59.724+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2019-12-10T21:07:59.724+05:30 [Info] Indexer Data Size Mean 35264996985 (32.8431G)
2019-12-10T21:07:59.724+05:30 [Info] Indexer Data Size Deviation 2225485751 (2.07265G) (6.31%)
2019-12-10T21:07:59.724+05:30 [Info] Total Index Data (from non-deleted node) 0
2019-12-10T21:07:59.724+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2019-12-10T21:07:59.724+05:30 [Info] No. Index (from non-deleted node) 0
2019-12-10T21:07:59.724+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2019/12/10 21:07:59 -------------------------------------------
2019/12/10 21:07:59 initial placement - 20-50M, 30 index, 3 replica, 4x
2019-12-10T21:08:00.487+05:30 [Info] Score: 0.03288915350368014
2019-12-10T21:08:00.487+05:30 [Info] Memory Quota: 113500337412 (105.705G)
2019-12-10T21:08:00.487+05:30 [Info] CPU Quota: 24
2019-12-10T21:08:00.487+05:30 [Info] Indexer Memory Mean 72930101061 (67.9214G)
2019-12-10T21:08:00.487+05:30 [Info] Indexer Memory Deviation 2398609288 (2.23388G) (3.29%)
2019-12-10T21:08:00.487+05:30 [Info] Indexer Memory Utilization 0.6426
2019-12-10T21:08:00.487+05:30 [Info] Indexer CPU Mean 21.6665
2019-12-10T21:08:00.487+05:30 [Info] Indexer CPU Deviation 4.12 (19.02%)
2019-12-10T21:08:00.487+05:30 [Info] Indexer CPU Utilization 0.9028
2019-12-10T21:08:00.487+05:30 [Info] Indexer IO Mean 0.0000
2019-12-10T21:08:00.487+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2019-12-10T21:08:00.487+05:30 [Info] Indexer Drain Rate Mean 0.0000
2019-12-10T21:08:00.487+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2019-12-10T21:08:00.487+05:30 [Info] Indexer Scan Rate Mean 0.0000
2019-12-10T21:08:00.487+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2019-12-10T21:08:00.487+05:30 [Info] Indexer Data Size Mean 72930101061 (67.9214G)
2019-12-10T21:08:00.487+05:30 [Info] Indexer Data Size Deviation 2398609288 (2.23388G) (3.29%)
2019-12-10T21:08:00.487+05:30 [Info] Total Index Data (from non-deleted node) 0
2019-12-10T21:08:00.487+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2019-12-10T21:08:00.487+05:30 [Info] No. Index (from non-deleted node) 0
2019-12-10T21:08:00.487+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2019/12/10 21:08:00 -------------------------------------------
2019/12/10 21:08:00 initial placement - 200-500M, 10 index, 3 replica, 2x
2019-12-10T21:08:00.684+05:30 [Info] Score: 0.06101877694431854
2019-12-10T21:08:00.684+05:30 [Info] Memory Quota: 450402847172 (419.47G)
2019-12-10T21:08:00.684+05:30 [Info] CPU Quota: 12
2019-12-10T21:08:00.684+05:30 [Info] Indexer Memory Mean 337480855553 (314.304G)
2019-12-10T21:08:00.684+05:30 [Info] Indexer Memory Deviation 20592669048 (19.1784G) (6.10%)
2019-12-10T21:08:00.684+05:30 [Info] Indexer Memory Utilization 0.7493
2019-12-10T21:08:00.684+05:30 [Info] Indexer CPU Mean 9.5446
2019-12-10T21:08:00.684+05:30 [Info] Indexer CPU Deviation 2.08 (21.78%)
2019-12-10T21:08:00.684+05:30 [Info] Indexer CPU Utilization 0.7954
2019-12-10T21:08:00.684+05:30 [Info] Indexer IO Mean 0.0000
2019-12-10T21:08:00.684+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2019-12-10T21:08:00.684+05:30 [Info] Indexer Drain Rate Mean 0.0000
2019-12-10T21:08:00.684+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2019-12-10T21:08:00.684+05:30 [Info] Indexer Scan Rate Mean 0.0000
2019-12-10T21:08:00.684+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2019-12-10T21:08:00.684+05:30 [Info] Indexer Data Size Mean 337480855553 (314.304G)
2019-12-10T21:08:00.684+05:30 [Info] Indexer Data Size Deviation 20592669048 (19.1784G) (6.10%)
2019-12-10T21:08:00.684+05:30 [Info] Total Index Data (from non-deleted node) 0
2019-12-10T21:08:00.684+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2019-12-10T21:08:00.684+05:30 [Info] No. Index (from non-deleted node) 0
2019-12-10T21:08:00.684+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2019/12/10 21:08:00 -------------------------------------------
2019/12/10 21:08:00 initial placement - 200-500M, 30 index, 3 replica, 2x
2019-12-10T21:08:03.533+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T21:08:03.567+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T21:08:03.567+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T21:08:03.568+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T21:08:03.568+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T21:08:03.570+05:30 [Info] switched currmeta from 455 -> 455 force true 
2019-12-10T21:08:03.574+05:30 [Info] switched currmeta from 451 -> 451 force true 
2019-12-10T21:08:04.285+05:30 [Info] Score: 0.04265100185028441
2019-12-10T21:08:04.285+05:30 [Info] Memory Quota: 505570010974 (470.849G)
2019-12-10T21:08:04.285+05:30 [Info] CPU Quota: 12
2019-12-10T21:08:04.285+05:30 [Info] Indexer Memory Mean 401348680759 (373.785G)
2019-12-10T21:08:04.285+05:30 [Info] Indexer Memory Deviation 17117923325 (15.9423G) (4.27%)
2019-12-10T21:08:04.285+05:30 [Info] Indexer Memory Utilization 0.7939
2019-12-10T21:08:04.285+05:30 [Info] Indexer CPU Mean 10.9400
2019-12-10T21:08:04.285+05:30 [Info] Indexer CPU Deviation 2.65 (24.26%)
2019-12-10T21:08:04.285+05:30 [Info] Indexer CPU Utilization 0.9117
2019-12-10T21:08:04.286+05:30 [Info] Indexer IO Mean 0.0000
2019-12-10T21:08:04.286+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2019-12-10T21:08:04.286+05:30 [Info] Indexer Drain Rate Mean 0.0000
2019-12-10T21:08:04.286+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2019-12-10T21:08:04.286+05:30 [Info] Indexer Scan Rate Mean 0.0000
2019-12-10T21:08:04.286+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2019-12-10T21:08:04.286+05:30 [Info] Indexer Data Size Mean 401348680759 (373.785G)
2019-12-10T21:08:04.286+05:30 [Info] Indexer Data Size Deviation 17117923325 (15.9423G) (4.27%)
2019-12-10T21:08:04.286+05:30 [Info] Total Index Data (from non-deleted node) 0
2019-12-10T21:08:04.286+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2019-12-10T21:08:04.286+05:30 [Info] No. Index (from non-deleted node) 0
2019-12-10T21:08:04.286+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2019/12/10 21:08:04 -------------------------------------------
2019/12/10 21:08:04 initial placement - mixed small/medium, 30 index, 3 replica, 1.5/4x
2019-12-10T21:08:05.561+05:30 [Info] Score: 0.014389813061804623
2019-12-10T21:08:05.561+05:30 [Info] Memory Quota: 320286738870 (298.29G)
2019-12-10T21:08:05.561+05:30 [Info] CPU Quota: 16
2019-12-10T21:08:05.561+05:30 [Info] Indexer Memory Mean 253003691140 (235.628G)
2019-12-10T21:08:05.561+05:30 [Info] Indexer Memory Deviation 3640675819 (3.39064G) (1.44%)
2019-12-10T21:08:05.561+05:30 [Info] Indexer Memory Utilization 0.7899
2019-12-10T21:08:05.561+05:30 [Info] Indexer CPU Mean 7.7522
2019-12-10T21:08:05.561+05:30 [Info] Indexer CPU Deviation 3.35 (43.19%)
2019-12-10T21:08:05.561+05:30 [Info] Indexer CPU Utilization 0.4845
2019-12-10T21:08:05.561+05:30 [Info] Indexer IO Mean 0.0000
2019-12-10T21:08:05.561+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2019-12-10T21:08:05.561+05:30 [Info] Indexer Drain Rate Mean 0.0000
2019-12-10T21:08:05.561+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2019-12-10T21:08:05.561+05:30 [Info] Indexer Scan Rate Mean 0.0000
2019-12-10T21:08:05.561+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2019-12-10T21:08:05.561+05:30 [Info] Indexer Data Size Mean 253003691140 (235.628G)
2019-12-10T21:08:05.561+05:30 [Info] Indexer Data Size Deviation 3640675819 (3.39064G) (1.44%)
2019-12-10T21:08:05.561+05:30 [Info] Total Index Data (from non-deleted node) 0
2019-12-10T21:08:05.561+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2019-12-10T21:08:05.561+05:30 [Info] No. Index (from non-deleted node) 0
2019-12-10T21:08:05.561+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2019/12/10 21:08:05 -------------------------------------------
2019/12/10 21:08:05 initial placement - mixed all, 30 index, 3 replica, 1.5/4x
2019-12-10T21:08:06.630+05:30 [Info] Score: 0.018004484598246606
2019-12-10T21:08:06.630+05:30 [Info] Memory Quota: 426780927771 (397.471G)
2019-12-10T21:08:06.630+05:30 [Info] CPU Quota: 24
2019-12-10T21:08:06.630+05:30 [Info] Indexer Memory Mean 343003479696 (319.447G)
2019-12-10T21:08:06.630+05:30 [Info] Indexer Memory Deviation 6175600867 (5.75148G) (1.80%)
2019-12-10T21:08:06.630+05:30 [Info] Indexer Memory Utilization 0.8037
2019-12-10T21:08:06.630+05:30 [Info] Indexer CPU Mean 10.4233
2019-12-10T21:08:06.630+05:30 [Info] Indexer CPU Deviation 5.29 (50.77%)
2019-12-10T21:08:06.630+05:30 [Info] Indexer CPU Utilization 0.4343
2019-12-10T21:08:06.630+05:30 [Info] Indexer IO Mean 0.0000
2019-12-10T21:08:06.630+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2019-12-10T21:08:06.630+05:30 [Info] Indexer Drain Rate Mean 0.0000
2019-12-10T21:08:06.630+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2019-12-10T21:08:06.630+05:30 [Info] Indexer Scan Rate Mean 0.0000
2019-12-10T21:08:06.630+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2019-12-10T21:08:06.630+05:30 [Info] Indexer Data Size Mean 343003479696 (319.447G)
2019-12-10T21:08:06.630+05:30 [Info] Indexer Data Size Deviation 6175600867 (5.75148G) (1.80%)
2019-12-10T21:08:06.630+05:30 [Info] Total Index Data (from non-deleted node) 0
2019-12-10T21:08:06.630+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2019-12-10T21:08:06.630+05:30 [Info] No. Index (from non-deleted node) 0
2019-12-10T21:08:06.630+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2019/12/10 21:08:06 -------------------------------------------
2019/12/10 21:08:06 initial placement - 6 2M index, 1 replica, 2x
2019-12-10T21:08:06.807+05:30 [Info] Score: 0
2019-12-10T21:08:06.807+05:30 [Info] Memory Quota: 4848128000 (4.51517G)
2019-12-10T21:08:06.807+05:30 [Info] CPU Quota: 2
2019-12-10T21:08:06.807+05:30 [Info] Indexer Memory Mean 2080000000 (1.93715G)
2019-12-10T21:08:06.807+05:30 [Info] Indexer Memory Deviation 0 (0) (0.00%)
2019-12-10T21:08:06.807+05:30 [Info] Indexer Memory Utilization 0.4290
2019-12-10T21:08:06.808+05:30 [Info] Indexer CPU Mean 1.2000
2019-12-10T21:08:06.808+05:30 [Info] Indexer CPU Deviation 0.00 (0.00%)
2019-12-10T21:08:06.808+05:30 [Info] Indexer CPU Utilization 0.6000
2019-12-10T21:08:06.808+05:30 [Info] Indexer IO Mean 0.0000
2019-12-10T21:08:06.808+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2019-12-10T21:08:06.808+05:30 [Info] Indexer Drain Rate Mean 0.0000
2019-12-10T21:08:06.808+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2019-12-10T21:08:06.808+05:30 [Info] Indexer Scan Rate Mean 0.0000
2019-12-10T21:08:06.808+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2019-12-10T21:08:06.808+05:30 [Info] Indexer Data Size Mean 2080000000 (1.93715G)
2019-12-10T21:08:06.808+05:30 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2019-12-10T21:08:06.808+05:30 [Info] Total Index Data (from non-deleted node) 0
2019-12-10T21:08:06.808+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2019-12-10T21:08:06.808+05:30 [Info] No. Index (from non-deleted node) 0
2019-12-10T21:08:06.808+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2019/12/10 21:08:06 -------------------------------------------
2019/12/10 21:08:06 initial placement - 5 20M primary index, 2 replica, 2x
2019-12-10T21:08:07.199+05:30 [Info] Score: 0
2019-12-10T21:08:07.199+05:30 [Info] Memory Quota: 14310128000 (13.3273G)
2019-12-10T21:08:07.199+05:30 [Info] CPU Quota: 2
2019-12-10T21:08:07.199+05:30 [Info] Indexer Memory Mean 10960000000 (10.2073G)
2019-12-10T21:08:07.199+05:30 [Info] Indexer Memory Deviation 0 (0) (0.00%)
2019-12-10T21:08:07.199+05:30 [Info] Indexer Memory Utilization 0.7659
2019-12-10T21:08:07.199+05:30 [Info] Indexer CPU Mean 1.2000
2019-12-10T21:08:07.199+05:30 [Info] Indexer CPU Deviation 0.00 (0.00%)
2019-12-10T21:08:07.199+05:30 [Info] Indexer CPU Utilization 0.6000
2019-12-10T21:08:07.199+05:30 [Info] Indexer IO Mean 0.0000
2019-12-10T21:08:07.199+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2019-12-10T21:08:07.199+05:30 [Info] Indexer Drain Rate Mean 0.0000
2019-12-10T21:08:07.199+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2019-12-10T21:08:07.199+05:30 [Info] Indexer Scan Rate Mean 0.0000
2019-12-10T21:08:07.199+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2019-12-10T21:08:07.199+05:30 [Info] Indexer Data Size Mean 10960000000 (10.2073G)
2019-12-10T21:08:07.199+05:30 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2019-12-10T21:08:07.199+05:30 [Info] Total Index Data (from non-deleted node) 0
2019-12-10T21:08:07.199+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2019-12-10T21:08:07.199+05:30 [Info] No. Index (from non-deleted node) 0
2019-12-10T21:08:07.199+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2019/12/10 21:08:07 -------------------------------------------
2019/12/10 21:08:07 initial placement - 5 20M array index, 2 replica, 2x
2019-12-10T21:08:07.595+05:30 [Info] Score: 0
2019-12-10T21:08:07.595+05:30 [Info] Memory Quota: 237416768000 (221.112G)
2019-12-10T21:08:07.595+05:30 [Info] CPU Quota: 2
2019-12-10T21:08:07.595+05:30 [Info] Indexer Memory Mean 191440000000 (178.292G)
2019-12-10T21:08:07.595+05:30 [Info] Indexer Memory Deviation 0 (0) (0.00%)
2019-12-10T21:08:07.595+05:30 [Info] Indexer Memory Utilization 0.8063
2019-12-10T21:08:07.595+05:30 [Info] Indexer CPU Mean 1.2000
2019-12-10T21:08:07.595+05:30 [Info] Indexer CPU Deviation 0.00 (0.00%)
2019-12-10T21:08:07.595+05:30 [Info] Indexer CPU Utilization 0.6000
2019-12-10T21:08:07.595+05:30 [Info] Indexer IO Mean 0.0000
2019-12-10T21:08:07.595+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2019-12-10T21:08:07.595+05:30 [Info] Indexer Drain Rate Mean 0.0000
2019-12-10T21:08:07.595+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2019-12-10T21:08:07.595+05:30 [Info] Indexer Scan Rate Mean 0.0000
2019-12-10T21:08:07.595+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2019-12-10T21:08:07.596+05:30 [Info] Indexer Data Size Mean 191440000000 (178.292G)
2019-12-10T21:08:07.596+05:30 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2019-12-10T21:08:07.596+05:30 [Info] Total Index Data (from non-deleted node) 0
2019-12-10T21:08:07.596+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2019-12-10T21:08:07.596+05:30 [Info] No. Index (from non-deleted node) 0
2019-12-10T21:08:07.596+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2019/12/10 21:08:07 -------------------------------------------
2019/12/10 21:08:07 initial placement - 3 replica constraint, 2 index, 2x
2019-12-10T21:08:07.981+05:30 [Info] Score: 0
2019-12-10T21:08:07.981+05:30 [Info] Memory Quota: 530294000 (505.728M)
2019-12-10T21:08:07.981+05:30 [Info] CPU Quota: 2
2019-12-10T21:08:07.981+05:30 [Info] Indexer Memory Mean 2600000 (2.47955M)
2019-12-10T21:08:07.981+05:30 [Info] Indexer Memory Deviation 0 (0) (0.00%)
2019-12-10T21:08:07.981+05:30 [Info] Indexer Memory Utilization 0.0049
2019-12-10T21:08:07.981+05:30 [Info] Indexer CPU Mean 0.0000
2019-12-10T21:08:07.981+05:30 [Info] Indexer CPU Deviation 0.00 (0.00%)
2019-12-10T21:08:07.981+05:30 [Info] Indexer CPU Utilization 0.0000
2019-12-10T21:08:07.981+05:30 [Info] Indexer IO Mean 0.0000
2019-12-10T21:08:07.981+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2019-12-10T21:08:07.981+05:30 [Info] Indexer Drain Rate Mean 0.0000
2019-12-10T21:08:07.981+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2019-12-10T21:08:07.981+05:30 [Info] Indexer Scan Rate Mean 0.0000
2019-12-10T21:08:07.981+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2019-12-10T21:08:07.981+05:30 [Info] Indexer Data Size Mean 2600000 (2.47955M)
2019-12-10T21:08:07.981+05:30 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2019-12-10T21:08:07.981+05:30 [Info] Total Index Data (from non-deleted node) 0
2019-12-10T21:08:07.981+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2019-12-10T21:08:07.981+05:30 [Info] No. Index (from non-deleted node) 0
2019-12-10T21:08:07.981+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2019/12/10 21:08:07 -------------------------------------------
2019/12/10 21:08:07 incr placement - 20-50M, 5 2M index, 1 replica, 1x
2019-12-10T21:08:10.192+05:30 [Info] Score: 0.014379877136943393
2019-12-10T21:08:10.192+05:30 [Info] Memory Quota: 125233041042 (116.632G)
2019-12-10T21:08:10.192+05:30 [Info] CPU Quota: 27
2019-12-10T21:08:10.192+05:30 [Info] Indexer Memory Mean 71117238485 (66.2331G)
2019-12-10T21:08:10.192+05:30 [Info] Indexer Memory Deviation 1022657151 (975.282M) (1.44%)
2019-12-10T21:08:10.192+05:30 [Info] Indexer Memory Utilization 0.5679
2019-12-10T21:08:10.192+05:30 [Info] Indexer CPU Mean 20.1734
2019-12-10T21:08:10.192+05:30 [Info] Indexer CPU Deviation 1.87 (9.29%)
2019-12-10T21:08:10.192+05:30 [Info] Indexer CPU Utilization 0.7472
2019-12-10T21:08:10.192+05:30 [Info] Indexer IO Mean 0.0000
2019-12-10T21:08:10.192+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2019-12-10T21:08:10.192+05:30 [Info] Indexer Drain Rate Mean 0.0000
2019-12-10T21:08:10.192+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2019-12-10T21:08:10.192+05:30 [Info] Indexer Scan Rate Mean 0.0000
2019-12-10T21:08:10.192+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2019-12-10T21:08:10.192+05:30 [Info] Indexer Data Size Mean 71117238485 (66.2331G)
2019-12-10T21:08:10.192+05:30 [Info] Indexer Data Size Deviation 1022657151 (975.282M) (1.44%)
2019-12-10T21:08:10.192+05:30 [Info] Total Index Data (from non-deleted node) 0
2019-12-10T21:08:10.192+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2019-12-10T21:08:10.192+05:30 [Info] No. Index (from non-deleted node) 0
2019-12-10T21:08:10.192+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2019/12/10 21:08:10 -------------------------------------------
2019/12/10 21:08:10 incr placement - mixed small/medium, 6 2M index, 1 replica, 1x
2019-12-10T21:08:13.898+05:30 [Info] Score: 0.0008743719299104943
2019-12-10T21:08:13.898+05:30 [Info] Memory Quota: 536870912000 (500G)
2019-12-10T21:08:13.898+05:30 [Info] CPU Quota: 20
2019-12-10T21:08:13.898+05:30 [Info] Indexer Memory Mean 393025602195 (366.034G)
2019-12-10T21:08:13.898+05:30 [Info] Indexer Memory Deviation 343650554 (327.731M) (0.09%)
2019-12-10T21:08:13.898+05:30 [Info] Indexer Memory Utilization 0.7321
2019-12-10T21:08:13.898+05:30 [Info] Indexer CPU Mean 14.2305
2019-12-10T21:08:13.898+05:30 [Info] Indexer CPU Deviation 0.95 (6.65%)
2019-12-10T21:08:13.898+05:30 [Info] Indexer CPU Utilization 0.7115
2019-12-10T21:08:13.898+05:30 [Info] Indexer IO Mean 0.0000
2019-12-10T21:08:13.898+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2019-12-10T21:08:13.898+05:30 [Info] Indexer Drain Rate Mean 0.0000
2019-12-10T21:08:13.898+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2019-12-10T21:08:13.898+05:30 [Info] Indexer Scan Rate Mean 0.0000
2019-12-10T21:08:13.898+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2019-12-10T21:08:13.898+05:30 [Info] Indexer Data Size Mean 393025602195 (366.034G)
2019-12-10T21:08:13.898+05:30 [Info] Indexer Data Size Deviation 343650554 (327.731M) (0.09%)
2019-12-10T21:08:13.898+05:30 [Info] Total Index Data (from non-deleted node) 0
2019-12-10T21:08:13.898+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2019-12-10T21:08:13.898+05:30 [Info] No. Index (from non-deleted node) 0
2019-12-10T21:08:13.898+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2019/12/10 21:08:13 -------------------------------------------
2019/12/10 21:08:13 incr placement - 3 server group, 3 replica, 1x
2019-12-10T21:08:14.188+05:30 [Info] Score: 0
2019-12-10T21:08:14.188+05:30 [Info] Memory Quota: 530294000 (505.728M)
2019-12-10T21:08:14.188+05:30 [Info] CPU Quota: 16
2019-12-10T21:08:14.188+05:30 [Info] Indexer Memory Mean 2600000 (2.47955M)
2019-12-10T21:08:14.188+05:30 [Info] Indexer Memory Deviation 0 (0) (0.00%)
2019-12-10T21:08:14.188+05:30 [Info] Indexer Memory Utilization 0.0049
2019-12-10T21:08:14.188+05:30 [Info] Indexer CPU Mean 0.0000
2019-12-10T21:08:14.188+05:30 [Info] Indexer CPU Deviation 0.00 (0.00%)
2019-12-10T21:08:14.188+05:30 [Info] Indexer CPU Utilization 0.0000
2019-12-10T21:08:14.188+05:30 [Info] Indexer IO Mean 0.0000
2019-12-10T21:08:14.188+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2019-12-10T21:08:14.188+05:30 [Info] Indexer Drain Rate Mean 0.0000
2019-12-10T21:08:14.188+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2019-12-10T21:08:14.188+05:30 [Info] Indexer Scan Rate Mean 0.0000
2019-12-10T21:08:14.188+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2019-12-10T21:08:14.188+05:30 [Info] Indexer Data Size Mean 2600000 (2.47955M)
2019-12-10T21:08:14.188+05:30 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2019-12-10T21:08:14.188+05:30 [Info] Total Index Data (from non-deleted node) 0
2019-12-10T21:08:14.188+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2019-12-10T21:08:14.188+05:30 [Info] No. Index (from non-deleted node) 0
2019-12-10T21:08:14.188+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2019/12/10 21:08:14 -------------------------------------------
2019/12/10 21:08:14 incr placement - 2 server group, 3 replica, 1x
2019-12-10T21:08:14.192+05:30 [Warn] Index has more replica than server group. Index=index1 0 (replica 2) Bucket=bucket2
2019-12-10T21:08:14.192+05:30 [Warn] Index has more replica than server group. Index=index1 0 Bucket=bucket2
2019-12-10T21:08:14.192+05:30 [Warn] Index has more replica than server group. Index=index1 0 (replica 1) Bucket=bucket2
2019-12-10T21:08:14.488+05:30 [Info] Score: 0
2019-12-10T21:08:14.488+05:30 [Info] Memory Quota: 530294000 (505.728M)
2019-12-10T21:08:14.488+05:30 [Info] CPU Quota: 16
2019-12-10T21:08:14.488+05:30 [Info] Indexer Memory Mean 2600000 (2.47955M)
2019-12-10T21:08:14.488+05:30 [Info] Indexer Memory Deviation 0 (0) (0.00%)
2019-12-10T21:08:14.488+05:30 [Info] Indexer Memory Utilization 0.0049
2019-12-10T21:08:14.488+05:30 [Info] Indexer CPU Mean 0.0000
2019-12-10T21:08:14.488+05:30 [Info] Indexer CPU Deviation 0.00 (0.00%)
2019-12-10T21:08:14.488+05:30 [Info] Indexer CPU Utilization 0.0000
2019-12-10T21:08:14.488+05:30 [Info] Indexer IO Mean 0.0000
2019-12-10T21:08:14.488+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2019-12-10T21:08:14.488+05:30 [Info] Indexer Drain Rate Mean 0.0000
2019-12-10T21:08:14.488+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2019-12-10T21:08:14.488+05:30 [Info] Indexer Scan Rate Mean 0.0000
2019-12-10T21:08:14.488+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2019-12-10T21:08:14.488+05:30 [Info] Indexer Data Size Mean 2600000 (2.47955M)
2019-12-10T21:08:14.488+05:30 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2019-12-10T21:08:14.488+05:30 [Info] Total Index Data (from non-deleted node) 0
2019-12-10T21:08:14.488+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2019-12-10T21:08:14.488+05:30 [Info] No. Index (from non-deleted node) 0
2019-12-10T21:08:14.488+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2019/12/10 21:08:14 -------------------------------------------
2019/12/10 21:08:14 rebalance - 20-50M, 90 index, 20% shuffle, 1x, utilization 90%+
2019-12-10T21:08:16.750+05:30 [Info] Score: 0.12698566735379485
2019-12-10T21:08:16.750+05:30 [Info] Memory Quota: 139586437120 (130G)
2019-12-10T21:08:16.750+05:30 [Info] CPU Quota: 30
2019-12-10T21:08:16.751+05:30 [Info] Indexer Memory Mean 88876568718 (82.7728G)
2019-12-10T21:08:16.751+05:30 [Info] Indexer Memory Deviation 3055739731 (2.84588G) (3.44%)
2019-12-10T21:08:16.751+05:30 [Info] Indexer Memory Utilization 0.6367
2019-12-10T21:08:16.751+05:30 [Info] Indexer CPU Mean 24.0538
2019-12-10T21:08:16.751+05:30 [Info] Indexer CPU Deviation 3.24 (13.49%)
2019-12-10T21:08:16.751+05:30 [Info] Indexer CPU Utilization 0.8018
2019-12-10T21:08:16.751+05:30 [Info] Indexer IO Mean 0.0000
2019-12-10T21:08:16.751+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2019-12-10T21:08:16.751+05:30 [Info] Indexer Drain Rate Mean 0.0000
2019-12-10T21:08:16.751+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2019-12-10T21:08:16.751+05:30 [Info] Indexer Scan Rate Mean 0.0000
2019-12-10T21:08:16.751+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2019-12-10T21:08:16.751+05:30 [Info] Indexer Data Size Mean 88876568718 (82.7728G)
2019-12-10T21:08:16.751+05:30 [Info] Indexer Data Size Deviation 3055739731 (2.84588G) (3.44%)
2019-12-10T21:08:16.751+05:30 [Info] Total Index Data (from non-deleted node) 993.273G
2019-12-10T21:08:16.751+05:30 [Info] Index Data Moved (exclude new node) 242.066G (24.37%)
2019-12-10T21:08:16.751+05:30 [Info] No. Index (from non-deleted node) 90
2019-12-10T21:08:16.751+05:30 [Info] No. Index Moved (exclude new node) 19 (21.11%)
2019/12/10 21:08:16 -------------------------------------------
2019/12/10 21:08:16 rebalance - mixed small/medium, 90 index, 20% shuffle, 1x
2019-12-10T21:08:19.466+05:30 [Info] Score: 0.13368088372329537
2019-12-10T21:08:19.466+05:30 [Info] Memory Quota: 536870912000 (500G)
2019-12-10T21:08:19.466+05:30 [Info] CPU Quota: 20
2019-12-10T21:08:19.466+05:30 [Info] Indexer Memory Mean 392505602195 (365.549G)
2019-12-10T21:08:19.466+05:30 [Info] Indexer Memory Deviation 21437764746 (19.9655G) (5.46%)
2019-12-10T21:08:19.466+05:30 [Info] Indexer Memory Utilization 0.7311
2019-12-10T21:08:19.466+05:30 [Info] Indexer CPU Mean 13.9305
2019-12-10T21:08:19.466+05:30 [Info] Indexer CPU Deviation 3.29 (23.58%)
2019-12-10T21:08:19.466+05:30 [Info] Indexer CPU Utilization 0.6965
2019-12-10T21:08:19.466+05:30 [Info] Indexer IO Mean 0.0000
2019-12-10T21:08:19.466+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2019-12-10T21:08:19.466+05:30 [Info] Indexer Drain Rate Mean 0.0000
2019-12-10T21:08:19.466+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2019-12-10T21:08:19.466+05:30 [Info] Indexer Scan Rate Mean 0.0000
2019-12-10T21:08:19.466+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2019-12-10T21:08:19.466+05:30 [Info] Indexer Data Size Mean 392505602195 (365.549G)
2019-12-10T21:08:19.466+05:30 [Info] Indexer Data Size Deviation 21437764746 (19.9655G) (5.46%)
2019-12-10T21:08:19.466+05:30 [Info] Total Index Data (from non-deleted node) 4.28378T
2019-12-10T21:08:19.466+05:30 [Info] Index Data Moved (exclude new node) 1.26163T (29.45%)
2019-12-10T21:08:19.466+05:30 [Info] No. Index (from non-deleted node) 90
2019-12-10T21:08:19.466+05:30 [Info] No. Index Moved (exclude new node) 14 (15.56%)
2019/12/10 21:08:19 -------------------------------------------
2019/12/10 21:08:19 rebalance - travel sample, 10% shuffle, 1x
2019-12-10T21:08:19.696+05:30 [Info] Score: 0.03458036016416904
2019-12-10T21:08:19.696+05:30 [Info] Memory Quota: 536870912 (512M)
2019-12-10T21:08:19.696+05:30 [Info] CPU Quota: 8
2019-12-10T21:08:19.696+05:30 [Info] Indexer Memory Mean 17503138 (16.6923M)
2019-12-10T21:08:19.696+05:30 [Info] Indexer Memory Deviation 1815794 (1.73168M) (10.37%)
2019-12-10T21:08:19.696+05:30 [Info] Indexer Memory Utilization 0.0326
2019-12-10T21:08:19.696+05:30 [Info] Indexer CPU Mean 0.0000
2019-12-10T21:08:19.696+05:30 [Info] Indexer CPU Deviation 0.00 (0.00%)
2019-12-10T21:08:19.696+05:30 [Info] Indexer CPU Utilization 0.0000
2019-12-10T21:08:19.696+05:30 [Info] Indexer IO Mean 0.0000
2019-12-10T21:08:19.696+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2019-12-10T21:08:19.696+05:30 [Info] Indexer Drain Rate Mean 0.0000
2019-12-10T21:08:19.696+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2019-12-10T21:08:19.696+05:30 [Info] Indexer Scan Rate Mean 0.0000
2019-12-10T21:08:19.696+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2019-12-10T21:08:19.696+05:30 [Info] Indexer Data Size Mean 0 (0)
2019-12-10T21:08:19.696+05:30 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2019-12-10T21:08:19.696+05:30 [Info] Total Index Data (from non-deleted node) 0
2019-12-10T21:08:19.696+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2019-12-10T21:08:19.696+05:30 [Info] No. Index (from non-deleted node) 10
2019-12-10T21:08:19.696+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2019/12/10 21:08:19 -------------------------------------------
2019/12/10 21:08:19 rebalance - 20-50M, 90 index, swap 2, 1x
2019-12-10T21:08:21.575+05:30 [Info] Score: 0.028161635375202734
2019-12-10T21:08:21.575+05:30 [Info] Memory Quota: 139586437120 (130G)
2019-12-10T21:08:21.575+05:30 [Info] CPU Quota: 30
2019-12-10T21:08:21.575+05:30 [Info] Indexer Memory Mean 88876568718 (82.7728G)
2019-12-10T21:08:21.575+05:30 [Info] Indexer Memory Deviation 2502909521 (2.33102G) (2.82%)
2019-12-10T21:08:21.575+05:30 [Info] Indexer Memory Utilization 0.6367
2019-12-10T21:08:21.576+05:30 [Info] Indexer CPU Mean 24.0538
2019-12-10T21:08:21.576+05:30 [Info] Indexer CPU Deviation 1.90 (7.89%)
2019-12-10T21:08:21.576+05:30 [Info] Indexer CPU Utilization 0.8018
2019-12-10T21:08:21.576+05:30 [Info] Indexer IO Mean 0.0000
2019-12-10T21:08:21.576+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2019-12-10T21:08:21.576+05:30 [Info] Indexer Drain Rate Mean 0.0000
2019-12-10T21:08:21.576+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2019-12-10T21:08:21.576+05:30 [Info] Indexer Scan Rate Mean 0.0000
2019-12-10T21:08:21.576+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2019-12-10T21:08:21.576+05:30 [Info] Indexer Data Size Mean 88876568718 (82.7728G)
2019-12-10T21:08:21.576+05:30 [Info] Indexer Data Size Deviation 2502909521 (2.33102G) (2.82%)
2019-12-10T21:08:21.576+05:30 [Info] Total Index Data (from non-deleted node) 0
2019-12-10T21:08:21.576+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2019-12-10T21:08:21.576+05:30 [Info] No. Index (from non-deleted node) 0
2019-12-10T21:08:21.576+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2019/12/10 21:08:21 -------------------------------------------
2019/12/10 21:08:21 rebalance - mixed small/medium, 90 index, swap 2, 1x
2019-12-10T21:08:23.264+05:30 [Info] Score: 0.002022823495622612
2019-12-10T21:08:23.264+05:30 [Info] Memory Quota: 536870912000 (500G)
2019-12-10T21:08:23.264+05:30 [Info] CPU Quota: 20
2019-12-10T21:08:23.264+05:30 [Info] Indexer Memory Mean 392505602195 (365.549G)
2019-12-10T21:08:23.264+05:30 [Info] Indexer Memory Deviation 793969554 (757.188M) (0.20%)
2019-12-10T21:08:23.264+05:30 [Info] Indexer Memory Utilization 0.7311
2019-12-10T21:08:23.264+05:30 [Info] Indexer CPU Mean 13.9305
2019-12-10T21:08:23.264+05:30 [Info] Indexer CPU Deviation 1.46 (10.47%)
2019-12-10T21:08:23.264+05:30 [Info] Indexer CPU Utilization 0.6965
2019-12-10T21:08:23.264+05:30 [Info] Indexer IO Mean 0.0000
2019-12-10T21:08:23.264+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2019-12-10T21:08:23.264+05:30 [Info] Indexer Drain Rate Mean 0.0000
2019-12-10T21:08:23.264+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2019-12-10T21:08:23.264+05:30 [Info] Indexer Scan Rate Mean 0.0000
2019-12-10T21:08:23.264+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2019-12-10T21:08:23.264+05:30 [Info] Indexer Data Size Mean 392505602195 (365.549G)
2019-12-10T21:08:23.264+05:30 [Info] Indexer Data Size Deviation 793969554 (757.188M) (0.20%)
2019-12-10T21:08:23.264+05:30 [Info] Total Index Data (from non-deleted node) 0
2019-12-10T21:08:23.264+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2019-12-10T21:08:23.264+05:30 [Info] No. Index (from non-deleted node) 0
2019-12-10T21:08:23.264+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2019/12/10 21:08:23 -------------------------------------------
2019/12/10 21:08:23 rebalance - travel sample, swap 2, 1x
2019-12-10T21:08:25.135+05:30 [Info] Score: 0.0006385854742565169
2019-12-10T21:08:25.135+05:30 [Info] Memory Quota: 536870912 (512M)
2019-12-10T21:08:25.135+05:30 [Info] CPU Quota: 8
2019-12-10T21:08:25.135+05:30 [Info] Indexer Memory Mean 17503138 (16.6923M)
2019-12-10T21:08:25.135+05:30 [Info] Indexer Memory Deviation 22354 (21.8301K) (0.13%)
2019-12-10T21:08:25.135+05:30 [Info] Indexer Memory Utilization 0.0326
2019-12-10T21:08:25.135+05:30 [Info] Indexer CPU Mean 0.0000
2019-12-10T21:08:25.135+05:30 [Info] Indexer CPU Deviation 0.00 (0.00%)
2019-12-10T21:08:25.135+05:30 [Info] Indexer CPU Utilization 0.0000
2019-12-10T21:08:25.135+05:30 [Info] Indexer IO Mean 0.0000
2019-12-10T21:08:25.135+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2019-12-10T21:08:25.135+05:30 [Info] Indexer Drain Rate Mean 0.0000
2019-12-10T21:08:25.135+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2019-12-10T21:08:25.135+05:30 [Info] Indexer Scan Rate Mean 0.0000
2019-12-10T21:08:25.135+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2019-12-10T21:08:25.135+05:30 [Info] Indexer Data Size Mean 0 (0)
2019-12-10T21:08:25.135+05:30 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2019-12-10T21:08:25.135+05:30 [Info] Total Index Data (from non-deleted node) 0
2019-12-10T21:08:25.135+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2019-12-10T21:08:25.135+05:30 [Info] No. Index (from non-deleted node) 0
2019-12-10T21:08:25.135+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2019/12/10 21:08:25 -------------------------------------------
2019/12/10 21:08:25 rebalance - 8 identical index, add 4, 1x
2019-12-10T21:08:25.764+05:30 [Info] Score: 0
2019-12-10T21:08:25.764+05:30 [Info] Memory Quota: 530294000 (505.728M)
2019-12-10T21:08:25.764+05:30 [Info] CPU Quota: 2
2019-12-10T21:08:25.764+05:30 [Info] Indexer Memory Mean 2600000 (2.47955M)
2019-12-10T21:08:25.764+05:30 [Info] Indexer Memory Deviation 0 (0) (0.00%)
2019-12-10T21:08:25.764+05:30 [Info] Indexer Memory Utilization 0.0049
2019-12-10T21:08:25.764+05:30 [Info] Indexer CPU Mean 0.0000
2019-12-10T21:08:25.764+05:30 [Info] Indexer CPU Deviation 0.00 (0.00%)
2019-12-10T21:08:25.764+05:30 [Info] Indexer CPU Utilization 0.0000
2019-12-10T21:08:25.764+05:30 [Info] Indexer IO Mean 0.0000
2019-12-10T21:08:25.764+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2019-12-10T21:08:25.764+05:30 [Info] Indexer Drain Rate Mean 0.0000
2019-12-10T21:08:25.764+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2019-12-10T21:08:25.764+05:30 [Info] Indexer Scan Rate Mean 0.0000
2019-12-10T21:08:25.764+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2019-12-10T21:08:25.764+05:30 [Info] Indexer Data Size Mean 2600000 (2.47955M)
2019-12-10T21:08:25.764+05:30 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2019-12-10T21:08:25.764+05:30 [Info] Total Index Data (from non-deleted node) 19.8364M
2019-12-10T21:08:25.764+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2019-12-10T21:08:25.764+05:30 [Info] No. Index (from non-deleted node) 8
2019-12-10T21:08:25.764+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2019/12/10 21:08:25 -------------------------------------------
2019/12/10 21:08:25 rebalance - 8 identical index, delete 2, 2x
2019-12-10T21:08:26.178+05:30 [Info] Score: 0
2019-12-10T21:08:26.178+05:30 [Info] Memory Quota: 1060588000 (1011.46M)
2019-12-10T21:08:26.178+05:30 [Info] CPU Quota: 4
2019-12-10T21:08:26.178+05:30 [Info] Indexer Memory Mean 10400000 (9.91821M)
2019-12-10T21:08:26.178+05:30 [Info] Indexer Memory Deviation 0 (0) (0.00%)
2019-12-10T21:08:26.178+05:30 [Info] Indexer Memory Utilization 0.0098
2019-12-10T21:08:26.178+05:30 [Info] Indexer CPU Mean 0.0000
2019-12-10T21:08:26.178+05:30 [Info] Indexer CPU Deviation 0.00 (0.00%)
2019-12-10T21:08:26.178+05:30 [Info] Indexer CPU Utilization 0.0000
2019-12-10T21:08:26.178+05:30 [Info] Indexer IO Mean 0.0000
2019-12-10T21:08:26.178+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2019-12-10T21:08:26.178+05:30 [Info] Indexer Drain Rate Mean 0.0000
2019-12-10T21:08:26.178+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2019-12-10T21:08:26.178+05:30 [Info] Indexer Scan Rate Mean 0.0000
2019-12-10T21:08:26.178+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2019-12-10T21:08:26.178+05:30 [Info] Indexer Data Size Mean 10400000 (9.91821M)
2019-12-10T21:08:26.178+05:30 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2019-12-10T21:08:26.178+05:30 [Info] Total Index Data (from non-deleted node) 0
2019-12-10T21:08:26.178+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2019-12-10T21:08:26.178+05:30 [Info] No. Index (from non-deleted node) 0
2019-12-10T21:08:26.178+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2019/12/10 21:08:26 -------------------------------------------
2019/12/10 21:08:26 rebalance - drop replcia - 3 replica, 3 zone, delete 1, 2x
2019-12-10T21:08:26.179+05:30 [Warn] There is more replia than available nodes.  Will not move index replica (default,country) from ejected node 127.0.0.1:9001
2019-12-10T21:08:26.179+05:30 [Info] Score: 0
2019-12-10T21:08:26.179+05:30 [Info] Memory Quota: 536870912 (512M)
2019-12-10T21:08:26.179+05:30 [Info] CPU Quota: 16
2019-12-10T21:08:26.179+05:30 [Info] Indexer Memory Mean 0 (0)
2019-12-10T21:08:26.179+05:30 [Info] Indexer Memory Deviation 0 (0) (0.00%)
2019-12-10T21:08:26.179+05:30 [Info] Indexer Memory Utilization 0.0000
2019-12-10T21:08:26.179+05:30 [Info] Indexer CPU Mean 0.0000
2019-12-10T21:08:26.179+05:30 [Info] Indexer CPU Deviation 0.00 (0.00%)
2019-12-10T21:08:26.179+05:30 [Info] Indexer CPU Utilization 0.0000
2019-12-10T21:08:26.179+05:30 [Info] Indexer IO Mean 0.0000
2019-12-10T21:08:26.180+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2019-12-10T21:08:26.180+05:30 [Info] Indexer Drain Rate Mean 0.0000
2019-12-10T21:08:26.180+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2019-12-10T21:08:26.180+05:30 [Info] Indexer Scan Rate Mean 0.0000
2019-12-10T21:08:26.180+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2019-12-10T21:08:26.180+05:30 [Info] Indexer Data Size Mean 0 (0)
2019-12-10T21:08:26.180+05:30 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2019-12-10T21:08:26.180+05:30 [Info] Total Index Data (from non-deleted node) 0
2019-12-10T21:08:26.180+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2019-12-10T21:08:26.180+05:30 [Info] No. Index (from non-deleted node) 0
2019-12-10T21:08:26.180+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2019/12/10 21:08:26 -------------------------------------------
2019/12/10 21:08:26 rebalance - rebuid replica - 3 replica, 3 zone, add 1, delete 1, 1x
2019-12-10T21:08:26.180+05:30 [Info] Score: 0
2019-12-10T21:08:26.180+05:30 [Info] Memory Quota: 268435456 (256M)
2019-12-10T21:08:26.180+05:30 [Info] CPU Quota: 8
2019-12-10T21:08:26.180+05:30 [Info] Indexer Memory Mean 0 (0)
2019-12-10T21:08:26.180+05:30 [Info] Indexer Memory Deviation 0 (0) (0.00%)
2019-12-10T21:08:26.180+05:30 [Info] Indexer Memory Utilization 0.0000
2019-12-10T21:08:26.180+05:30 [Info] Indexer CPU Mean 0.0000
2019-12-10T21:08:26.180+05:30 [Info] Indexer CPU Deviation 0.00 (0.00%)
2019-12-10T21:08:26.180+05:30 [Info] Indexer CPU Utilization 0.0000
2019-12-10T21:08:26.180+05:30 [Info] Indexer IO Mean 0.0000
2019-12-10T21:08:26.180+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2019-12-10T21:08:26.180+05:30 [Info] Indexer Drain Rate Mean 0.0000
2019-12-10T21:08:26.180+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2019-12-10T21:08:26.180+05:30 [Info] Indexer Scan Rate Mean 0.0000
2019-12-10T21:08:26.180+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2019-12-10T21:08:26.180+05:30 [Info] Indexer Data Size Mean 0 (0)
2019-12-10T21:08:26.180+05:30 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2019-12-10T21:08:26.180+05:30 [Info] Total Index Data (from non-deleted node) 0
2019-12-10T21:08:26.180+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2019-12-10T21:08:26.180+05:30 [Info] No. Index (from non-deleted node) 0
2019-12-10T21:08:26.180+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2019/12/10 21:08:26 -------------------------------------------
2019/12/10 21:08:26 Minimum memory test 1: min memory = 0
2019/12/10 21:08:26 -------------------------------------------
2019/12/10 21:08:26 Minimum memory test 2: min memory > quota
2019-12-10T21:08:26.215+05:30 [Info] Planner::Fail to create plan satisyfig constraint. Re-planning. Num of Try=1.  Elapsed Time=7us
2019-12-10T21:08:26.215+05:30 [Info] Planner::Fail to create plan satisyfig constraint. Re-planning. Num of Try=2.  Elapsed Time=8us
2019-12-10T21:08:26.215+05:30 [Info] Planner::Fail to create plan satisyfig constraint. Re-planning. Num of Try=3.  Elapsed Time=7us
2019/12/10 21:08:36 -------------------------------------------
2019/12/10 21:08:36 Minimum memory test 3: min memory < quota
2019-12-10T21:08:38.006+05:30 [Info] connected with 1 indexers
2019-12-10T21:08:38.006+05:30 [Info] index 11365992841125477121 has replicas [7609328433260783641]: 
2019-12-10T21:08:38.006+05:30 [Info] index 11365992841125477121 has equivalents [11365992841125477121]: 
2019-12-10T21:08:38.006+05:30 [Info] client load stats {"7609328433260783641": 2.684639505965332e+09}
2019-12-10T21:08:38.006+05:30 [Info] client hit stats {"7609328433260783641": 12}
2019-12-10T21:08:38.006+05:30 [Info] client pending item stats {"7609328433260783641": 9223372036854775807}
2019-12-10T21:08:38.006+05:30 [Info] client stats current {"7609328433260783641": true}
2019-12-10T21:08:38.009+05:30 [Info] num concurrent scans {0}
2019-12-10T21:08:38.009+05:30 [Info] average scan response {2669 ms}
2019-12-10T21:08:38.009+05:30 [Info] GSIC[default/default-1575991477896699991] logstats "default" {"gsi_scan_count":109,"gsi_scan_duration":49572622052,"gsi_throttle_duration":568290857,"gsi_prime_duration":5633895208,"gsi_blocked_duration":129970833,"gsi_total_temp_files":0}
2019-12-10T21:08:38.391+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] active conns 0, free conns 1
2019/12/10 21:08:47 -------------------------------------------
2019/12/10 21:08:47 Minimum memory test 4: replica repair with min memory > quota
2019-12-10T21:08:47.677+05:30 [Info] Rebuilding lost replica for (default,country,0)
2019-12-10T21:08:47.677+05:30 [Info] Planner::Fail to create plan satisyfig constraint. Re-planning. Num of Try=1.  Elapsed Time=7us
2019-12-10T21:08:47.677+05:30 [Info] Planner::Fail to create plan satisyfig constraint. Re-planning. Num of Try=2.  Elapsed Time=6us
2019-12-10T21:08:47.677+05:30 [Info] Planner::Fail to create plan satisyfig constraint. Re-planning. Num of Try=3.  Elapsed Time=4us
2019-12-10T21:08:47.678+05:30 [Info] Planner::Fail to create plan satisyfig constraint. Re-planning. Num of Try=4.  Elapsed Time=111us
2019-12-10T21:08:47.678+05:30 [Info] Planner::Fail to create plan satisyfig constraint. Re-planning. Num of Try=5.  Elapsed Time=92us
2019-12-10T21:08:47.678+05:30 [Info] Cannot rebuild lost replica due to resource constraint in cluster.  Will not rebuild lost replica.
2019-12-10T21:08:47.678+05:30 [Warn] 
MemoryQuota: 200
CpuQuota: 8
--- Violations for index  (mem 130, cpu 0) at node 127.0.0.1:9003 
	Cannot move to 127.0.0.1:9001: ReplicaViolation (free mem 1.67772e+07T, free cpu 8)
	Cannot move to 127.0.0.1:9002: ReplicaViolation (free mem 1.67772e+07T, free cpu 8)
2019/12/10 21:08:52 -------------------------------------------
2019/12/10 21:08:52 Minimum memory test 5: replica repair with min memory < quota
2019-12-10T21:08:52.126+05:30 [Info] Rebuilding lost replica for (default,country,0)
2019/12/10 21:08:52 -------------------------------------------
2019/12/10 21:08:52 Minimum memory test 6: rebalance with min memory > quota
2019-12-10T21:08:52.243+05:30 [Info] Planner::Fail to create plan satisyfig constraint. Re-planning. Num of Try=1.  Elapsed Time=8us
2019-12-10T21:08:52.243+05:30 [Info] Planner::Fail to create plan satisyfig constraint. Re-planning. Num of Try=2.  Elapsed Time=7us
2019-12-10T21:08:52.243+05:30 [Info] Planner::Fail to create plan satisyfig constraint. Re-planning. Num of Try=3.  Elapsed Time=7us
2019-12-10T21:08:52.243+05:30 [Info] Planner::Fail to create plan satisyfig constraint. Re-planning. Num of Try=4.  Elapsed Time=6us
2019-12-10T21:08:52.243+05:30 [Info] Planner::Fail to create plan satisyfig constraint. Re-planning. Num of Try=5.  Elapsed Time=6us
2019-12-10T21:08:52.243+05:30 [Info] Planner::Fail to create plan satisyfig constraint. Re-planning. Num of Try=6.  Elapsed Time=7us
2019-12-10T21:08:52.243+05:30 [Info] Planner::Fail to create plan satisyfig constraint. Re-planning. Num of Try=7.  Elapsed Time=6us
2019-12-10T21:08:52.243+05:30 [Info] Planner::Fail to create plan satisyfig constraint. Re-planning. Num of Try=8.  Elapsed Time=6us
2019-12-10T21:08:52.243+05:30 [Info] Planner::Fail to create plan satisyfig constraint. Re-planning. Num of Try=9.  Elapsed Time=6us
2019-12-10T21:08:52.243+05:30 [Info] Planner::Fail to create plan satisyfig constraint. Re-planning. Num of Try=10.  Elapsed Time=6us
2019-12-10T21:08:52.243+05:30 [Info] Planner::Fail to create plan satisyfig constraint. Re-planning. Num of Try=11.  Elapsed Time=7us
2019-12-10T21:08:52.243+05:30 [Info] Planner::Fail to create plan satisyfig constraint. Re-planning. Num of Try=12.  Elapsed Time=6us
2019/12/10 21:08:52 -------------------------------------------
2019/12/10 21:08:52 Minimum memory test 7: rebalance-out with min memory > quota
2019-12-10T21:08:52.244+05:30 [Info] Planner::Fail to create plan satisyfig constraint. Re-planning. Num of Try=1.  Elapsed Time=5us
2019-12-10T21:08:52.244+05:30 [Info] Planner::Fail to create plan satisyfig constraint. Re-planning. Num of Try=2.  Elapsed Time=7us
2019-12-10T21:08:52.244+05:30 [Info] Planner::Fail to create plan satisyfig constraint. Re-planning. Num of Try=3.  Elapsed Time=7us
2019-12-10T21:08:52.244+05:30 [Info] Planner::Fail to create plan satisyfig constraint. Re-planning. Num of Try=4.  Elapsed Time=6us
2019-12-10T21:08:52.244+05:30 [Info] Planner::Fail to create plan satisyfig constraint. Re-planning. Num of Try=5.  Elapsed Time=31us
2019-12-10T21:08:52.244+05:30 [Info] Planner::Fail to create plan satisyfig constraint. Re-planning. Num of Try=6.  Elapsed Time=7us
2019-12-10T21:08:52.244+05:30 [Warn] Unable to find a solution with rersource costraint.  Relax resource constraint check.
2019/12/10 21:08:52 -------------------------------------------
2019/12/10 21:08:52 Minimum memory test 8: plan with min memory > quota
2019-12-10T21:08:52.255+05:30 [Info] Planner::Fail to create plan satisyfig constraint. Re-planning. Num of Try=1.  Elapsed Time=872us
2019-12-10T21:08:52.255+05:30 [Info] Planner::Fail to create plan satisyfig constraint. Re-planning. Num of Try=2.  Elapsed Time=7us
2019-12-10T21:08:52.255+05:30 [Info] Planner::Fail to create plan satisyfig constraint. Re-planning. Num of Try=3.  Elapsed Time=7us
2019-12-10T21:08:52.255+05:30 [Info] Planner::Fail to create plan satisyfig constraint. Re-planning. Num of Try=4.  Elapsed Time=6us
2019-12-10T21:08:52.255+05:30 [Info] Planner::Fail to create plan satisyfig constraint. Re-planning. Num of Try=5.  Elapsed Time=6us
2019-12-10T21:08:52.255+05:30 [Info] Planner::Fail to create plan satisyfig constraint. Re-planning. Num of Try=6.  Elapsed Time=7us
2019-12-10T21:08:52.255+05:30 [Warn] Unable to find a solution with rersource costraint.  Relax resource constraint check.
2019/12/10 21:08:52 -------------------------------------------
2019/12/10 21:08:52 Minimum memory test 9: single node rebalance with min memory > quota
2019-12-10T21:08:52.256+05:30 [Info] Planner::Fail to create plan satisyfig constraint. Re-planning. Num of Try=1.  Elapsed Time=5us
2019-12-10T21:08:52.256+05:30 [Info] Planner::Fail to create plan satisyfig constraint. Re-planning. Num of Try=2.  Elapsed Time=7us
2019-12-10T21:08:52.256+05:30 [Info] Planner::Fail to create plan satisyfig constraint. Re-planning. Num of Try=3.  Elapsed Time=6us
2019-12-10T21:08:52.256+05:30 [Info] Planner::Fail to create plan satisyfig constraint. Re-planning. Num of Try=4.  Elapsed Time=6us
2019-12-10T21:08:52.256+05:30 [Info] Planner::Fail to create plan satisyfig constraint. Re-planning. Num of Try=5.  Elapsed Time=5us
2019-12-10T21:08:52.256+05:30 [Info] Planner::Fail to create plan satisyfig constraint. Re-planning. Num of Try=6.  Elapsed Time=6us
2019-12-10T21:08:52.256+05:30 [Info] Planner::Fail to create plan satisyfig constraint. Re-planning. Num of Try=7.  Elapsed Time=8us
2019-12-10T21:08:52.256+05:30 [Info] Planner::Fail to create plan satisyfig constraint. Re-planning. Num of Try=8.  Elapsed Time=7us
2019-12-10T21:08:52.256+05:30 [Info] Planner::Fail to create plan satisyfig constraint. Re-planning. Num of Try=9.  Elapsed Time=7us
2019-12-10T21:08:52.256+05:30 [Info] Planner::Fail to create plan satisyfig constraint. Re-planning. Num of Try=10.  Elapsed Time=7us
2019-12-10T21:08:52.256+05:30 [Info] Planner::Fail to create plan satisyfig constraint. Re-planning. Num of Try=11.  Elapsed Time=6us
2019-12-10T21:08:52.256+05:30 [Info] Planner::Fail to create plan satisyfig constraint. Re-planning. Num of Try=12.  Elapsed Time=7us
2019/12/10 21:08:52 -------------------------------------------
2019/12/10 21:08:52 Minimum memory test 10: plan with partitioned index on empty cluster
--- PASS: TestPlanner (56.53s)
=== RUN   TestRestfulAPI
2019/12/10 21:08:52 In TestRestfulAPI()
2019/12/10 21:08:52 In DropAllSecondaryIndexes()
2019/12/10 21:08:52 Index found:  indexmut_1
2019/12/10 21:08:52 Dropped index indexmut_1
2019/12/10 21:08:53 Setting JSON docs in KV
2019/12/10 21:08:54 GET all indexes
2019/12/10 21:08:54 200 OK
2019/12/10 21:08:54 FOUND indexes: []
2019/12/10 21:08:54 DROP index: badindexid
2019/12/10 21:08:54 status: 400 Bad Request
2019/12/10 21:08:54 DROP index: 23544142
2019/12/10 21:08:54 status: 500 Internal Server Error
2019/12/10 21:08:54 TEST: malformed body
2019/12/10 21:08:54 400 Bad Request "invalid request body ({name:), unmarshal failed invalid character 'n' looking for beginning of object key string"

2019/12/10 21:08:54 TEST: missing field ``name``
2019/12/10 21:08:54 400 Bad Request "missing field name"
2019/12/10 21:08:54 TEST: empty field ``name``
2019/12/10 21:08:54 400 Bad Request "empty field name"
2019/12/10 21:08:54 TEST: missing field ``bucket``
2019/12/10 21:08:54 400 Bad Request "missing field bucket"
2019/12/10 21:08:54 TEST: empty field ``bucket``
2019/12/10 21:08:54 400 Bad Request "empty field bucket"
2019/12/10 21:08:54 TEST: missing field ``secExprs``
2019/12/10 21:08:54 400 Bad Request "missing field secExprs"
2019/12/10 21:08:54 TEST: empty field ``secExprs``
2019/12/10 21:08:54 400 Bad Request "empty field secExprs"
2019/12/10 21:08:54 TEST: incomplete field ``desc``
2019/12/10 21:08:54 400 Bad Request "incomplete desc information [true]"
2019/12/10 21:08:54 TEST: invalid field ``desc``
2019/12/10 21:08:54 400 Bad Request "incomplete desc information [1]"
2019/12/10 21:08:54 
2019/12/10 21:08:54 CREATE INDEX: idx1
2019/12/10 21:09:12 status : 201 Created
2019/12/10 21:09:12 {"id": "8032682156695677316"} 
2019/12/10 21:09:12 CREATE INDEX: idx2 (defer)
2019/12/10 21:09:12 status : 201 Created
2019/12/10 21:09:12 {"id": "10326807869293822692"} 
2019/12/10 21:09:12 CREATE INDEX: idx3 (defer)
2019/12/10 21:09:13 status : 201 Created
2019/12/10 21:09:13 {"id": "14167274238896899336"} 
2019/12/10 21:09:13 CREATE INDEX: idx4 (defer)
2019/12/10 21:09:13 status : 201 Created
2019/12/10 21:09:13 {"id": "570058471923651078"} 
2019/12/10 21:09:13 CREATE INDEX: idx5
2019/12/10 21:09:26 status : 201 Created
2019/12/10 21:09:26 {"id": "7025606987348794764"} 
2019/12/10 21:09:26 BUILD single deferred index
2019/12/10 21:09:26 202 Accepted
2019/12/10 21:09:26 GET all indexes
2019/12/10 21:09:26 200 OK
2019/12/10 21:09:26 index idx1 in INDEX_STATE_ACTIVE
2019/12/10 21:09:26 GET all indexes
2019/12/10 21:09:26 200 OK
2019/12/10 21:09:26 index idx2 in INDEX_STATE_INITIAL
2019/12/10 21:09:28 GET all indexes
2019/12/10 21:09:28 200 OK
2019/12/10 21:09:28 index idx2 in INDEX_STATE_INITIAL
2019/12/10 21:09:29 GET all indexes
2019/12/10 21:09:29 200 OK
2019/12/10 21:09:29 index idx2 in INDEX_STATE_INITIAL
2019/12/10 21:09:30 GET all indexes
2019/12/10 21:09:30 200 OK
2019/12/10 21:09:30 index idx2 in INDEX_STATE_INITIAL
2019/12/10 21:09:31 GET all indexes
2019/12/10 21:09:31 200 OK
2019/12/10 21:09:31 index idx2 in INDEX_STATE_INITIAL
2019/12/10 21:09:32 GET all indexes
2019/12/10 21:09:32 200 OK
2019/12/10 21:09:32 index idx2 in INDEX_STATE_INITIAL
2019/12/10 21:09:33 GET all indexes
2019/12/10 21:09:33 200 OK
2019/12/10 21:09:33 index idx2 in INDEX_STATE_INITIAL
2019/12/10 21:09:34 GET all indexes
2019/12/10 21:09:34 200 OK
2019/12/10 21:09:34 index idx2 in INDEX_STATE_INITIAL
2019/12/10 21:09:35 GET all indexes
2019/12/10 21:09:35 200 OK
2019/12/10 21:09:35 index idx2 in INDEX_STATE_INITIAL
2019/12/10 21:09:36 GET all indexes
2019/12/10 21:09:36 200 OK
2019/12/10 21:09:36 index idx2 in INDEX_STATE_INITIAL
2019/12/10 21:09:37 GET all indexes
2019/12/10 21:09:37 200 OK
2019/12/10 21:09:37 index idx2 in INDEX_STATE_INITIAL
2019/12/10 21:09:38 GET all indexes
2019/12/10 21:09:38 200 OK
2019/12/10 21:09:38 index idx2 in INDEX_STATE_INITIAL
2019/12/10 21:09:39 GET all indexes
2019/12/10 21:09:39 200 OK
2019/12/10 21:09:39 index idx2 in INDEX_STATE_INITIAL
2019/12/10 21:09:40 GET all indexes
2019/12/10 21:09:40 200 OK
2019/12/10 21:09:40 index idx2 in INDEX_STATE_INITIAL
2019/12/10 21:09:42 GET all indexes
2019/12/10 21:09:42 200 OK
2019/12/10 21:09:42 index idx2 in INDEX_STATE_INITIAL
2019/12/10 21:09:43 GET all indexes
2019/12/10 21:09:43 200 OK
2019/12/10 21:09:43 index idx2 in INDEX_STATE_INITIAL
2019/12/10 21:09:44 GET all indexes
2019/12/10 21:09:44 200 OK
2019/12/10 21:09:44 index idx2 in INDEX_STATE_ACTIVE
2019/12/10 21:09:44 BUILD many deferred index
2019/12/10 21:09:44 202 Accepted 
2019/12/10 21:09:44 GET all indexes
2019/12/10 21:09:44 200 OK
2019/12/10 21:09:44 index idx1 in INDEX_STATE_ACTIVE
2019/12/10 21:09:44 GET all indexes
2019/12/10 21:09:44 200 OK
2019/12/10 21:09:44 index idx2 in INDEX_STATE_ACTIVE
2019/12/10 21:09:44 GET all indexes
2019/12/10 21:09:44 200 OK
2019/12/10 21:09:44 index idx3 in INDEX_STATE_INITIAL
2019/12/10 21:09:45 GET all indexes
2019/12/10 21:09:45 200 OK
2019/12/10 21:09:45 index idx3 in INDEX_STATE_INITIAL
2019/12/10 21:09:46 GET all indexes
2019/12/10 21:09:46 200 OK
2019/12/10 21:09:46 index idx3 in INDEX_STATE_INITIAL
2019/12/10 21:09:47 GET all indexes
2019/12/10 21:09:47 200 OK
2019/12/10 21:09:47 index idx3 in INDEX_STATE_INITIAL
2019/12/10 21:09:48 GET all indexes
2019/12/10 21:09:48 200 OK
2019/12/10 21:09:48 index idx3 in INDEX_STATE_INITIAL
2019/12/10 21:09:49 GET all indexes
2019/12/10 21:09:49 200 OK
2019/12/10 21:09:49 index idx3 in INDEX_STATE_INITIAL
2019/12/10 21:09:51 GET all indexes
2019/12/10 21:09:51 200 OK
2019/12/10 21:09:51 index idx3 in INDEX_STATE_INITIAL
2019/12/10 21:09:52 GET all indexes
2019/12/10 21:09:52 200 OK
2019/12/10 21:09:52 index idx3 in INDEX_STATE_INITIAL
2019/12/10 21:09:53 GET all indexes
2019/12/10 21:09:53 200 OK
2019/12/10 21:09:53 index idx3 in INDEX_STATE_INITIAL
2019/12/10 21:09:54 GET all indexes
2019/12/10 21:09:54 200 OK
2019/12/10 21:09:54 index idx3 in INDEX_STATE_INITIAL
2019/12/10 21:09:55 GET all indexes
2019/12/10 21:09:55 200 OK
2019/12/10 21:09:55 index idx3 in INDEX_STATE_INITIAL
2019/12/10 21:09:56 GET all indexes
2019/12/10 21:09:56 200 OK
2019/12/10 21:09:56 index idx3 in INDEX_STATE_INITIAL
2019/12/10 21:09:57 GET all indexes
2019/12/10 21:09:57 200 OK
2019/12/10 21:09:57 index idx3 in INDEX_STATE_INITIAL
2019/12/10 21:09:58 GET all indexes
2019/12/10 21:09:58 200 OK
2019/12/10 21:09:58 index idx3 in INDEX_STATE_INITIAL
2019/12/10 21:09:59 GET all indexes
2019/12/10 21:09:59 200 OK
2019/12/10 21:09:59 index idx3 in INDEX_STATE_INITIAL
2019/12/10 21:10:00 GET all indexes
2019/12/10 21:10:00 200 OK
2019/12/10 21:10:00 index idx3 in INDEX_STATE_INITIAL
2019/12/10 21:10:01 GET all indexes
2019/12/10 21:10:01 200 OK
2019/12/10 21:10:01 index idx3 in INDEX_STATE_INITIAL
2019/12/10 21:10:02 GET all indexes
2019/12/10 21:10:02 200 OK
2019/12/10 21:10:02 index idx3 in INDEX_STATE_INITIAL
2019/12/10 21:10:04 GET all indexes
2019/12/10 21:10:04 200 OK
2019/12/10 21:10:04 index idx3 in INDEX_STATE_INITIAL
2019/12/10 21:10:05 GET all indexes
2019/12/10 21:10:05 200 OK
2019/12/10 21:10:05 index idx3 in INDEX_STATE_INITIAL
2019/12/10 21:10:06 GET all indexes
2019/12/10 21:10:06 200 OK
2019/12/10 21:10:06 index idx3 in INDEX_STATE_ACTIVE
2019/12/10 21:10:06 GET all indexes
2019/12/10 21:10:06 200 OK
2019/12/10 21:10:06 index idx4 in INDEX_STATE_ACTIVE
2019/12/10 21:10:06 GET all indexes
2019/12/10 21:10:06 200 OK
2019/12/10 21:10:06 index idx5 in INDEX_STATE_ACTIVE
2019/12/10 21:10:06 GET all indexes
2019/12/10 21:10:06 200 OK
2019/12/10 21:10:06 CREATED indexes: [8032682156695677316 10326807869293822692 14167274238896899336 570058471923651078 7025606987348794764]
2019/12/10 21:10:06 
2019/12/10 21:10:06 LOOKUP missing index
2019/12/10 21:10:06 status : 404 Not Found
2019/12/10 21:10:06 LOOKUP Pyongyang
2019/12/10 21:10:06 status : 200 OK
2019/12/10 21:10:06 number of entries 579
2019/12/10 21:10:06 Expected and Actual scan responses are the same
2019/12/10 21:10:06 LOOKUP with stale as false
2019/12/10 21:10:06 status : 200 OK
2019/12/10 21:10:06 number of entries 579
2019/12/10 21:10:06 Expected and Actual scan responses are the same
2019/12/10 21:10:06 LOOKUP with Rome
2019/12/10 21:10:06 status : 200 OK
2019/12/10 21:10:06 number of entries 552
2019/12/10 21:10:06 Expected and Actual scan responses are the same
2019/12/10 21:10:06 RANGE missing index
2019/12/10 21:10:06 Status : 404 Not Found
2019/12/10 21:10:06 RANGE cities - none
2019/12/10 21:10:06 Status : 200 OK
2019/12/10 21:10:12 number of entries 140902
2019/12/10 21:10:13 Expected and Actual scan responses are the same
2019/12/10 21:10:13 RANGE cities -low
2019/12/10 21:10:13 Status : 200 OK
2019/12/10 21:10:19 number of entries 140902
2019/12/10 21:10:20 Expected and Actual scan responses are the same
2019/12/10 21:10:20 RANGE cities -high
2019/12/10 21:10:20 Status : 200 OK
2019/12/10 21:10:26 number of entries 140902
2019/12/10 21:10:26 Expected and Actual scan responses are the same
2019/12/10 21:10:26 RANGE cities - both
2019/12/10 21:10:26 Status : 200 OK
2019/12/10 21:10:32 number of entries 140902
2019/12/10 21:10:33 Expected and Actual scan responses are the same
2019/12/10 21:10:33 RANGE missing cities
2019/12/10 21:10:33 Status : 200 OK
2019/12/10 21:10:33 number of entries 0
2019/12/10 21:10:33 Expected and Actual scan responses are the same
2019/12/10 21:10:33 
2019/12/10 21:10:33 SCANALL missing index
2019/12/10 21:10:33 {"limit":1000000,"stale":"ok"}
2019/12/10 21:10:33 Status : 404 Not Found
2019/12/10 21:10:33 SCANALL stale ok
2019/12/10 21:10:33 {"limit":1000000,"stale":"ok"}
2019/12/10 21:10:33 Status : 200 OK
2019/12/10 21:10:40 number of entries 140902
2019/12/10 21:10:41 Expected and Actual scan responses are the same
2019/12/10 21:10:41 SCANALL stale false
2019/12/10 21:10:41 {"limit":1000000,"stale":"false"}
2019/12/10 21:10:41 Status : 200 OK
2019/12/10 21:10:47 number of entries 140902
2019/12/10 21:10:48 Expected and Actual scan responses are the same
2019/12/10 21:10:48 
2019/12/10 21:10:48 COUNT missing index
2019/12/10 21:10:48 Status : 404 Not Found
2019/12/10 21:10:48 COUNT cities - none
2019/12/10 21:10:50 Status : 200 OK
2019/12/10 21:10:50 number of entries 140902
2019/12/10 21:10:50 COUNT cities -low
2019/12/10 21:10:53 Status : 200 OK
2019/12/10 21:10:53 number of entries 140902
2019/12/10 21:10:53 COUNT cities -high
2019/12/10 21:10:55 Status : 200 OK
2019/12/10 21:10:55 number of entries 140902
2019/12/10 21:10:55 COUNT cities - both
2019/12/10 21:10:58 Status : 200 OK
2019/12/10 21:10:58 number of entries 140902
2019/12/10 21:10:58 COUNT missing cities
2019/12/10 21:10:58 Status : 200 OK
2019/12/10 21:10:58 number of entries 0
2019/12/10 21:10:58 
2019/12/10 21:10:59 STATS: Testing URLs with valid authentication
2019/12/10 21:10:59 STATS: Testing URLs with invalid authentication
2019/12/10 21:10:59 STATS: Testing invalid URLs
2019/12/10 21:10:59 STATS: Testing unsupported methods
2019/12/10 21:10:59 
--- PASS: TestRestfulAPI (126.53s)
=== RUN   TestBucketDefaultDelete
2019/12/10 21:11:01 Deleted bucket default
2019/12/10 21:11:31 Created bucket default
2019/12/10 21:12:02 Populating the default bucket
2019/12/10 21:12:11 Using n1ql client
2019-12-10T21:12:11.340+05:30 [Info] GsiClient::UpdateUsecjson: using collatejson as data format between indexer and GsiClient
2019-12-10T21:12:11.340+05:30 [Info] GSIC[default/default-1575992531337069731] started ...
2019/12/10 21:12:11 Scan failed as expected with error: GSI index index_isActive not found.
2019/12/10 21:12:11 Populating the default bucket after it was deleted
2019/12/10 21:12:22 Created the secondary index index_isActive. Waiting for it become active
2019/12/10 21:12:22 Index is now active
2019/12/10 21:12:22 Using n1ql client
2019/12/10 21:12:22 Expected and Actual scan responses are the same
--- PASS: TestBucketDefaultDelete (83.11s)
=== RUN   TestMixedDatatypesScanAll
2019/12/10 21:12:22 In TestMixedDatatypesScanAll()
2019/12/10 21:12:22 Before test begin: Length of kv docs is 10002
2019/12/10 21:12:22 In DropAllSecondaryIndexes()
2019/12/10 21:12:22 Index found:  index_isActive
2019/12/10 21:12:22 Dropped index index_isActive
2019/12/10 21:12:22 Number of number fields is: 273
2019/12/10 21:12:22 Number of string fields is: 229
2019/12/10 21:12:22 Number of json fields is: 242
2019/12/10 21:12:22 Number of true bool fields is: 132
2019/12/10 21:12:22 Number of false bool fields is: 124
2019/12/10 21:12:22 After generate docs: Length of kv docs is 11002
2019/12/10 21:12:22 Setting mixed datatypes JSON docs in KV
2019/12/10 21:12:27 Created the secondary index index_mixeddt. Waiting for it become active
2019/12/10 21:12:27 Index is now active
2019/12/10 21:12:27 Using n1ql client
2019/12/10 21:12:27 Expected and Actual scan responses are the same
2019/12/10 21:12:27 Lengths of expected and actual scan results are:  1000 and 1000
2019/12/10 21:12:27 End: Length of kv docs is 11002
--- PASS: TestMixedDatatypesScanAll (5.43s)
=== RUN   TestMixedDatatypesRange_Float
2019/12/10 21:12:27 In TestMixedDatatypesRange_Float()
2019/12/10 21:12:27 In DropAllSecondaryIndexes()
2019/12/10 21:12:27 Index found:  index_mixeddt
2019/12/10 21:12:27 Dropped index index_mixeddt
2019/12/10 21:12:28 Number of number fields is: 227
2019/12/10 21:12:28 Number of string fields is: 258
2019/12/10 21:12:28 Number of json fields is: 271
2019/12/10 21:12:28 Number of true bool fields is: 135
2019/12/10 21:12:28 Number of false bool fields is: 109
2019/12/10 21:12:28 Setting mixed datatypes JSON docs in KV
2019/12/10 21:12:33 Created the secondary index index_mixeddt. Waiting for it become active
2019/12/10 21:12:33 Index is now active
2019/12/10 21:12:33 Using n1ql client
2019/12/10 21:12:33 Expected and Actual scan responses are the same
2019/12/10 21:12:33 Lengths of expected and actual scan results are:  19 and 19
2019/12/10 21:12:33 Using n1ql client
2019/12/10 21:12:33 Expected and Actual scan responses are the same
2019/12/10 21:12:33 Lengths of expected and actual scan results are:  2 and 2
2019/12/10 21:12:33 Length of kv docs is 12002
--- PASS: TestMixedDatatypesRange_Float (5.34s)
=== RUN   TestMixedDatatypesRange_String
2019/12/10 21:12:33 In TestMixedDatatypesRange_String()
2019/12/10 21:12:33 In DropAllSecondaryIndexes()
2019/12/10 21:12:33 Index found:  index_mixeddt
2019/12/10 21:12:33 Dropped index index_mixeddt
2019/12/10 21:12:33 Number of number fields is: 232
2019/12/10 21:12:33 Number of string fields is: 255
2019/12/10 21:12:33 Number of json fields is: 237
2019/12/10 21:12:33 Number of true bool fields is: 145
2019/12/10 21:12:33 Number of false bool fields is: 131
2019/12/10 21:12:33 Setting mixed datatypes JSON docs in KV
2019/12/10 21:12:38 Created the secondary index index_mixeddt. Waiting for it become active
2019/12/10 21:12:38 Index is now active
2019/12/10 21:12:38 Using n1ql client
2019/12/10 21:12:38 Expected and Actual scan responses are the same
2019/12/10 21:12:38 Lengths of expected and actual scan results are:  197 and 197
2019/12/10 21:12:38 Length of kv docs is 13002
--- PASS: TestMixedDatatypesRange_String (5.53s)
=== RUN   TestMixedDatatypesRange_Json
2019/12/10 21:12:38 In TestMixedDatatypesRange_Json()
2019/12/10 21:12:38 In DropAllSecondaryIndexes()
2019/12/10 21:12:38 Index found:  index_mixeddt
2019/12/10 21:12:38 Dropped index index_mixeddt
2019/12/10 21:12:38 Number of number fields is: 229
2019/12/10 21:12:38 Number of string fields is: 269
2019/12/10 21:12:38 Number of json fields is: 260
2019/12/10 21:12:38 Number of true bool fields is: 109
2019/12/10 21:12:38 Number of false bool fields is: 133
2019/12/10 21:12:38 Setting mixed datatypes JSON docs in KV
2019/12/10 21:12:43 Created the secondary index index_mixeddt. Waiting for it become active
2019/12/10 21:12:43 Index is now active
2019/12/10 21:12:43 Using n1ql client
2019/12/10 21:12:44 Expected and Actual scan responses are the same
2019/12/10 21:12:44 Lengths of expected and actual scan results are:  768 and 768
2019/12/10 21:12:44 Length of kv docs is 14002
--- PASS: TestMixedDatatypesRange_Json (5.38s)
=== RUN   TestMixedDatatypesScan_Bool
2019/12/10 21:12:44 In TestMixedDatatypesScan_Bool()
2019/12/10 21:12:44 In DropAllSecondaryIndexes()
2019/12/10 21:12:44 Index found:  index_mixeddt
2019/12/10 21:12:44 Dropped index index_mixeddt
2019/12/10 21:12:44 Number of number fields is: 247
2019/12/10 21:12:44 Number of string fields is: 254
2019/12/10 21:12:44 Number of json fields is: 244
2019/12/10 21:12:44 Number of true bool fields is: 134
2019/12/10 21:12:44 Number of false bool fields is: 121
2019/12/10 21:12:44 Setting mixed datatypes JSON docs in KV
2019/12/10 21:12:49 Created the secondary index index_mixeddt. Waiting for it become active
2019/12/10 21:12:49 Index is now active
2019/12/10 21:12:49 Using n1ql client
2019/12/10 21:12:49 Expected and Actual scan responses are the same
2019/12/10 21:12:49 Lengths of expected and actual scan results are:  523 and 523
2019/12/10 21:12:49 Using n1ql client
2019/12/10 21:12:49 Expected and Actual scan responses are the same
2019/12/10 21:12:49 Lengths of expected and actual scan results are:  494 and 494
2019/12/10 21:12:49 Length of kv docs is 15002
--- PASS: TestMixedDatatypesScan_Bool (5.68s)
=== RUN   TestLargeSecondaryKeyLength
2019/12/10 21:12:49 In TestLargeSecondaryKeyLength()
2019/12/10 21:12:49 In DropAllSecondaryIndexes()
2019/12/10 21:12:49 Index found:  index_mixeddt
2019/12/10 21:12:49 Dropped index index_mixeddt
2019/12/10 21:12:50 Setting JSON docs in KV
2019/12/10 21:12:55 Created the secondary index index_LongSecField. Waiting for it become active
2019/12/10 21:12:55 Index is now active
2019/12/10 21:12:55 Using n1ql client
2019/12/10 21:12:55 ScanAll: Lengths of expected and actual scan results are:  1000 and 1000
2019/12/10 21:12:55 Expected and Actual scan responses are the same
2019/12/10 21:12:55 Using n1ql client
2019/12/10 21:12:55 Range: Lengths of expected and actual scan results are:  814 and 814
2019/12/10 21:12:55 Expected and Actual scan responses are the same
2019/12/10 21:12:55 End: Length of kv docs is 16002
--- PASS: TestLargeSecondaryKeyLength (5.87s)
=== RUN   TestLargePrimaryKeyLength
2019/12/10 21:12:55 In TestLargePrimaryKeyLength()
2019/12/10 21:12:55 In DropAllSecondaryIndexes()
2019/12/10 21:12:55 Index found:  index_LongSecField
2019/12/10 21:12:55 Dropped index index_LongSecField
2019/12/10 21:12:55 Setting JSON docs in KV
2019/12/10 21:13:01 Created the secondary index index_LongPrimaryField. Waiting for it become active
2019/12/10 21:13:01 Index is now active
2019/12/10 21:13:01 Using n1ql client
2019/12/10 21:13:02 Lengths of num of docs and scanResults are:  17002 and 17002
2019/12/10 21:13:02 End: Length of kv docs is 17002
--- PASS: TestLargePrimaryKeyLength (6.86s)
=== RUN   TestUpdateMutations_DeleteField
2019/12/10 21:13:02 In TestUpdateMutations_DeleteField()
2019/12/10 21:13:02 Setting JSON docs in KV
2019/12/10 21:13:08 Created the secondary index index_bal. Waiting for it become active
2019/12/10 21:13:08 Index is now active
2019/12/10 21:13:09 Using n1ql client
2019/12/10 21:13:09 Expected and Actual scan responses are the same
2019/12/10 21:13:09 Using n1ql client
2019/12/10 21:13:09 Expected and Actual scan responses are the same
--- PASS: TestUpdateMutations_DeleteField (7.46s)
=== RUN   TestUpdateMutations_AddField
2019/12/10 21:13:09 In TestUpdateMutations_AddField()
2019/12/10 21:13:10 Setting JSON docs in KV
2019/12/10 21:13:16 Created the secondary index index_newField. Waiting for it become active
2019/12/10 21:13:16 Index is now active
2019/12/10 21:13:16 Using n1ql client
2019/12/10 21:13:16 Count of scan results before add field mutations:  0
2019/12/10 21:13:16 Expected and Actual scan responses are the same
2019/12/10 21:13:17 Using n1ql client
2019/12/10 21:13:17 Count of scan results after add field mutations:  300
2019/12/10 21:13:17 Expected and Actual scan responses are the same
--- PASS: TestUpdateMutations_AddField (7.35s)
=== RUN   TestUpdateMutations_DataTypeChange
2019/12/10 21:13:17 In TestUpdateMutations_DataTypeChange()
2019/12/10 21:13:17 Setting JSON docs in KV
2019/12/10 21:13:25 Created the secondary index index_isUserActive. Waiting for it become active
2019/12/10 21:13:25 Index is now active
2019/12/10 21:13:25 Using n1ql client
2019/12/10 21:13:26 Expected and Actual scan responses are the same
2019/12/10 21:13:26 Using n1ql client
2019/12/10 21:13:26 Expected and Actual scan responses are the same
2019/12/10 21:13:26 Using n1ql client
2019/12/10 21:13:26 Expected and Actual scan responses are the same
2019/12/10 21:13:26 Using n1ql client
2019/12/10 21:13:26 Expected and Actual scan responses are the same
--- PASS: TestUpdateMutations_DataTypeChange (9.76s)
=== RUN   TestMultipleBuckets
2019/12/10 21:13:26 In TestMultipleBuckets()
2019/12/10 21:13:26 In DropAllSecondaryIndexes()
2019/12/10 21:13:26 Index found:  index_LongPrimaryField
2019/12/10 21:13:27 Dropped index index_LongPrimaryField
2019/12/10 21:13:27 Index found:  index_bal
2019/12/10 21:13:27 Dropped index index_bal
2019/12/10 21:13:27 Index found:  index_newField
2019/12/10 21:13:27 Dropped index index_newField
2019/12/10 21:13:27 Index found:  index_isUserActive
2019/12/10 21:13:27 Dropped index index_isUserActive
2019/12/10 21:14:35 Flushed the bucket default
2019/12/10 21:14:38 Modified parameters of bucket default
2019/12/10 21:14:38 Created bucket testbucket2
2019/12/10 21:14:38 Created bucket testbucket3
2019/12/10 21:14:38 Created bucket testbucket4
2019/12/10 21:15:08 Generating docs and Populating all the buckets
2019/12/10 21:15:15 Created the secondary index bucket1_age. Waiting for it become active
2019/12/10 21:15:15 Index is now active
2019/12/10 21:15:23 Created the secondary index bucket2_city. Waiting for it become active
2019/12/10 21:15:23 Index is now active
2019/12/10 21:15:30 Created the secondary index bucket3_gender. Waiting for it become active
2019/12/10 21:15:30 Index is now active
2019/12/10 21:15:39 Created the secondary index bucket4_balance. Waiting for it become active
2019/12/10 21:15:39 Index is now active
2019/12/10 21:15:42 Using n1ql client
2019/12/10 21:15:42 Expected and Actual scan responses are the same
2019/12/10 21:15:42 Using n1ql client
2019-12-10T21:15:42.814+05:30 [Info] GsiClient::UpdateUsecjson: using collatejson as data format between indexer and GsiClient
2019-12-10T21:15:42.814+05:30 [Info] GSIC[default/testbucket2-1575992742810958913] started ...
2019/12/10 21:15:42 Expected and Actual scan responses are the same
2019/12/10 21:15:42 Using n1ql client
2019-12-10T21:15:42.829+05:30 [Info] GsiClient::UpdateUsecjson: using collatejson as data format between indexer and GsiClient
2019-12-10T21:15:42.829+05:30 [Info] GSIC[default/testbucket3-1575992742826774854] started ...
2019/12/10 21:15:42 Expected and Actual scan responses are the same
2019/12/10 21:15:42 Using n1ql client
2019-12-10T21:15:42.847+05:30 [Info] GsiClient::UpdateUsecjson: using collatejson as data format between indexer and GsiClient
2019-12-10T21:15:42.847+05:30 [Info] GSIC[default/testbucket4-1575992742845076991] started ...
2019/12/10 21:15:42 Expected and Actual scan responses are the same
2019/12/10 21:15:45 Deleted bucket testbucket2
2019/12/10 21:15:48 Deleted bucket testbucket3
2019/12/10 21:15:50 Deleted bucket testbucket4
2019/12/10 21:15:53 Modified parameters of bucket default
--- PASS: TestMultipleBuckets (176.76s)
=== RUN   TestBucketFlush
2019/12/10 21:16:23 In TestBucketFlush()
2019/12/10 21:16:23 In DropAllSecondaryIndexes()
2019/12/10 21:16:23 Index found:  bucket1_age
2019/12/10 21:16:23 Dropped index bucket1_age
2019/12/10 21:17:32 Flushed the bucket default
2019/12/10 21:17:37 Created the secondary index index_age. Waiting for it become active
2019/12/10 21:17:37 Index is now active
2019/12/10 21:17:37 Using n1ql client
2019/12/10 21:17:37 Expected and Actual scan responses are the same
2019/12/10 21:17:41 Created the secondary index index_gender. Waiting for it become active
2019/12/10 21:17:41 Index is now active
2019/12/10 21:17:41 Using n1ql client
2019/12/10 21:17:41 Expected and Actual scan responses are the same
2019/12/10 21:17:46 Created the secondary index index_city. Waiting for it become active
2019/12/10 21:17:46 Index is now active
2019/12/10 21:17:46 Using n1ql client
2019/12/10 21:17:47 Expected and Actual scan responses are the same
2019/12/10 21:18:56 Flushed the bucket default
2019/12/10 21:19:01 TestBucketFlush:: Flushed the bucket
2019/12/10 21:19:01 Using n1ql client
2019/12/10 21:19:01 Using n1ql client
2019/12/10 21:19:01 Using n1ql client
--- PASS: TestBucketFlush (157.55s)
=== RUN   TestLargeDocumentSize
2019/12/10 21:19:01 In TestLargeDocumentSize()
2019/12/10 21:19:01 Data file exists. Skipping download
2019/12/10 21:19:01 Length of docs and largeDocs = 200 and 200
2019/12/10 21:19:05 Created the secondary index index_userscreenname. Waiting for it become active
2019/12/10 21:19:05 Index is now active
2019/12/10 21:19:05 Using n1ql client
2019/12/10 21:19:06 Expected and Actual scan responses are the same
--- PASS: TestLargeDocumentSize (5.67s)
=== RUN   TestFieldsWithSpecialCharacters
2019/12/10 21:19:06 In TestFieldsWithSpecialCharacters()
2019/12/10 21:19:12 Created the secondary index index_specialchar. Waiting for it become active
2019/12/10 21:19:12 Index is now active
2019/12/10 21:19:12 Looking up for value ¶zx#š¶§
2019/12/10 21:19:12 Using n1ql client
2019/12/10 21:19:12 Expected and Actual scan responses are the same
--- PASS: TestFieldsWithSpecialCharacters (5.77s)
=== RUN   TestIndexNameValidation
2019/12/10 21:19:12 In TestIndexNameValidation()
2019/12/10 21:19:13 Setting JSON docs in KV
2019/12/10 21:19:14 Creation of index with invalid name ÌñÐÉx&(abc_% failed as expected
2019/12/10 21:19:18 Created the secondary index #primary-Index_test. Waiting for it become active
2019/12/10 21:19:18 Index is now active
2019/12/10 21:19:18 Using n1ql client
2019/12/10 21:19:19 Expected and Actual scan responses are the same
--- PASS: TestIndexNameValidation (6.98s)
=== RUN   TestSameFieldNameAtDifferentLevels
2019/12/10 21:19:19 In TestSameFieldNameAtDifferentLevels()
2019/12/10 21:19:19 Setting JSON docs in KV
2019/12/10 21:19:25 Created the secondary index cityindex. Waiting for it become active
2019/12/10 21:19:25 Index is now active
2019/12/10 21:19:25 Using n1ql client
2019/12/10 21:19:26 Expected and Actual scan responses are the same
--- PASS: TestSameFieldNameAtDifferentLevels (7.10s)
=== RUN   TestSameIndexNameInTwoBuckets
2019/12/10 21:19:26 In TestSameIndexNameInTwoBuckets()
2019/12/10 21:19:26 In DropAllSecondaryIndexes()
2019/12/10 21:19:26 Index found:  index_city
2019/12/10 21:19:26 Dropped index index_city
2019/12/10 21:19:26 Index found:  index_userscreenname
2019/12/10 21:19:27 Dropped index index_userscreenname
2019/12/10 21:19:27 Index found:  index_gender
2019/12/10 21:19:27 Dropped index index_gender
2019/12/10 21:19:27 Index found:  #primary-Index_test
2019/12/10 21:19:27 Dropped index #primary-Index_test
2019/12/10 21:19:27 Index found:  cityindex
2019/12/10 21:19:27 Dropped index cityindex
2019/12/10 21:19:27 Index found:  index_specialchar
2019/12/10 21:19:27 Dropped index index_specialchar
2019/12/10 21:19:27 Index found:  index_age
2019/12/10 21:19:28 Dropped index index_age
2019/12/10 21:20:36 Flushed the bucket default
2019/12/10 21:20:39 Modified parameters of bucket default
2019/12/10 21:20:39 Created bucket buck2
2019/12/10 21:21:09 Generating docs and Populating all the buckets
2019/12/10 21:21:15 Created the secondary index b_idx. Waiting for it become active
2019/12/10 21:21:15 Index is now active
2019/12/10 21:21:21 Created the secondary index b_idx. Waiting for it become active
2019/12/10 21:21:21 Index is now active
2019/12/10 21:21:24 Using n1ql client
2019/12/10 21:21:24 Expected and Actual scan responses are the same
2019/12/10 21:21:24 Using n1ql client
2019-12-10T21:21:24.477+05:30 [Info] GsiClient::UpdateUsecjson: using collatejson as data format between indexer and GsiClient
2019-12-10T21:21:24.477+05:30 [Info] GSIC[default/buck2-1575993084473641706] started ...
2019/12/10 21:21:24 Expected and Actual scan responses are the same
2019/12/10 21:21:27 Modified parameters of bucket default
2019/12/10 21:21:30 Deleted bucket buck2
--- PASS: TestSameIndexNameInTwoBuckets (153.25s)
=== RUN   TestLargeKeysSplChars
2019/12/10 21:22:00 Skipping test TestLargeKeysSplChars() for forestdb
--- PASS: TestLargeKeysSplChars (0.00s)
=== RUN   TestVeryLargeIndexKey
2019/12/10 21:22:00 Skipping test TestVeryLargeIndexKey() for forestdb
--- PASS: TestVeryLargeIndexKey (0.00s)
=== RUN   TestTempBufScanResult
2019/12/10 21:22:00 Skipping test TestTempBufScanResult() for forestdb
--- PASS: TestTempBufScanResult (0.00s)
=== RUN   TestBuildDeferredAnotherBuilding
2019/12/10 21:22:00 In TestBuildDeferredAnotherBuilding()
2019/12/10 21:22:00 In DropAllSecondaryIndexes()
2019/12/10 21:22:00 Index found:  b_idx
2019/12/10 21:22:00 Dropped index b_idx
2019/12/10 21:22:48 Setting JSON docs in KV
2019/12/10 21:25:07 Created the secondary index id_company
2019/12/10 21:25:08 Created the secondary index id_age
2019/12/10 21:25:09 Created the secondary index id_age1
2019/12/10 21:25:11 Build the deferred index id_age1. Waiting for the index to become active
2019/12/10 21:25:11 Waiting for index to go active ...
2019/12/10 21:25:12 Waiting for index to go active ...
2019/12/10 21:25:13 Waiting for index to go active ...
2019/12/10 21:25:14 Waiting for index to go active ...
2019/12/10 21:25:15 Waiting for index to go active ...
2019/12/10 21:25:16 Waiting for index to go active ...
2019/12/10 21:25:17 Waiting for index to go active ...
2019/12/10 21:25:18 Waiting for index to go active ...
2019/12/10 21:25:19 Waiting for index to go active ...
2019/12/10 21:25:20 Waiting for index to go active ...
2019/12/10 21:25:21 Waiting for index to go active ...
2019/12/10 21:25:22 Waiting for index to go active ...
2019/12/10 21:25:23 Waiting for index to go active ...
2019/12/10 21:25:24 Waiting for index to go active ...
2019/12/10 21:25:25 Waiting for index to go active ...
2019/12/10 21:25:26 Waiting for index to go active ...
2019/12/10 21:25:27 Waiting for index to go active ...
2019/12/10 21:25:28 Waiting for index to go active ...
2019/12/10 21:25:29 Waiting for index to go active ...
2019/12/10 21:25:30 Waiting for index to go active ...
2019/12/10 21:25:31 Waiting for index to go active ...
2019/12/10 21:25:32 Waiting for index to go active ...
2019/12/10 21:25:33 Waiting for index to go active ...
2019/12/10 21:25:34 Waiting for index to go active ...
2019/12/10 21:25:35 Waiting for index to go active ...
2019/12/10 21:25:36 Index is now active
2019/12/10 21:25:36 Build command issued for the deferred indexes [6786550346574332959]
2019/12/10 21:25:38 Build index failed as expected: Build index fails. Index id_age will retry building in the background for reason: Build Already In Progress. Bucket default.
2019/12/10 21:25:38 Waiting for index to go active ...
2019/12/10 21:25:39 Waiting for index to go active ...
2019/12/10 21:25:40 Waiting for index to go active ...
2019/12/10 21:25:41 Waiting for index to go active ...
2019/12/10 21:25:42 Waiting for index to go active ...
2019/12/10 21:25:43 Waiting for index to go active ...
2019/12/10 21:25:44 Waiting for index to go active ...
2019/12/10 21:25:45 Waiting for index to go active ...
2019/12/10 21:25:46 Waiting for index to go active ...
2019/12/10 21:25:47 Waiting for index to go active ...
2019/12/10 21:25:48 Waiting for index to go active ...
2019/12/10 21:25:49 Waiting for index to go active ...
2019/12/10 21:25:50 Waiting for index to go active ...
2019/12/10 21:25:51 Waiting for index to go active ...
2019/12/10 21:25:52 Waiting for index to go active ...
2019/12/10 21:25:53 Waiting for index to go active ...
2019/12/10 21:25:54 Waiting for index to go active ...
2019/12/10 21:25:55 Waiting for index to go active ...
2019/12/10 21:25:56 Waiting for index to go active ...
2019/12/10 21:25:57 Waiting for index to go active ...
2019/12/10 21:25:58 Waiting for index to go active ...
2019/12/10 21:25:59 Waiting for index to go active ...
2019/12/10 21:26:00 Waiting for index to go active ...
2019/12/10 21:26:01 Index is now active
2019/12/10 21:26:01 Waiting for index to go active ...
2019/12/10 21:26:02 Waiting for index to go active ...
2019/12/10 21:26:03 Waiting for index to go active ...
2019/12/10 21:26:04 Waiting for index to go active ...
2019/12/10 21:26:05 Waiting for index to go active ...
2019/12/10 21:26:06 Waiting for index to go active ...
2019/12/10 21:26:07 Waiting for index to go active ...
2019/12/10 21:26:08 Waiting for index to go active ...
2019/12/10 21:26:09 Waiting for index to go active ...
2019/12/10 21:26:10 Waiting for index to go active ...
2019/12/10 21:26:11 Waiting for index to go active ...
2019/12/10 21:26:12 Waiting for index to go active ...
2019/12/10 21:26:13 Waiting for index to go active ...
2019/12/10 21:26:14 Waiting for index to go active ...
2019/12/10 21:26:15 Waiting for index to go active ...
2019/12/10 21:26:16 Waiting for index to go active ...
2019/12/10 21:26:17 Waiting for index to go active ...
2019/12/10 21:26:18 Waiting for index to go active ...
2019/12/10 21:26:19 Waiting for index to go active ...
2019/12/10 21:26:20 Waiting for index to go active ...
2019/12/10 21:26:21 Waiting for index to go active ...
2019/12/10 21:26:22 Waiting for index to go active ...
2019/12/10 21:26:23 Waiting for index to go active ...
2019/12/10 21:26:24 Waiting for index to go active ...
2019/12/10 21:26:25 Waiting for index to go active ...
2019/12/10 21:26:26 Waiting for index to go active ...
2019/12/10 21:26:27 Waiting for index to go active ...
2019/12/10 21:26:28 Waiting for index to go active ...
2019/12/10 21:26:29 Waiting for index to go active ...
2019/12/10 21:26:30 Waiting for index to go active ...
2019/12/10 21:26:31 Waiting for index to go active ...
2019/12/10 21:26:32 Waiting for index to go active ...
2019/12/10 21:26:33 Waiting for index to go active ...
2019/12/10 21:26:34 Waiting for index to go active ...
2019/12/10 21:26:35 Waiting for index to go active ...
2019/12/10 21:26:36 Waiting for index to go active ...
2019/12/10 21:26:37 Waiting for index to go active ...
2019/12/10 21:26:38 Waiting for index to go active ...
2019/12/10 21:26:39 Waiting for index to go active ...
2019/12/10 21:26:40 Waiting for index to go active ...
2019/12/10 21:26:41 Waiting for index to go active ...
2019/12/10 21:26:42 Waiting for index to go active ...
2019/12/10 21:26:43 Waiting for index to go active ...
2019/12/10 21:26:44 Waiting for index to go active ...
2019/12/10 21:26:45 Waiting for index to go active ...
2019/12/10 21:26:46 Waiting for index to go active ...
2019/12/10 21:26:47 Waiting for index to go active ...
2019/12/10 21:26:48 Waiting for index to go active ...
2019/12/10 21:26:49 Waiting for index to go active ...
2019/12/10 21:26:50 Waiting for index to go active ...
2019/12/10 21:26:51 Waiting for index to go active ...
2019/12/10 21:26:52 Waiting for index to go active ...
2019/12/10 21:26:53 Waiting for index to go active ...
2019/12/10 21:26:54 Waiting for index to go active ...
2019/12/10 21:26:55 Waiting for index to go active ...
2019/12/10 21:26:56 Waiting for index to go active ...
2019/12/10 21:26:57 Waiting for index to go active ...
2019/12/10 21:26:58 Waiting for index to go active ...
2019/12/10 21:26:59 Waiting for index to go active ...
2019/12/10 21:27:00 Waiting for index to go active ...
2019/12/10 21:27:01 Waiting for index to go active ...
2019/12/10 21:27:02 Waiting for index to go active ...
2019/12/10 21:27:03 Waiting for index to go active ...
2019/12/10 21:27:04 Waiting for index to go active ...
2019/12/10 21:27:05 Waiting for index to go active ...
2019/12/10 21:27:06 Waiting for index to go active ...
2019/12/10 21:27:07 Waiting for index to go active ...
2019/12/10 21:27:08 Waiting for index to go active ...
2019/12/10 21:27:09 Waiting for index to go active ...
2019/12/10 21:27:10 Waiting for index to go active ...
2019/12/10 21:27:11 Waiting for index to go active ...
2019/12/10 21:27:12 Waiting for index to go active ...
2019/12/10 21:27:13 Waiting for index to go active ...
2019/12/10 21:27:14 Waiting for index to go active ...
2019/12/10 21:27:15 Waiting for index to go active ...
2019/12/10 21:27:16 Waiting for index to go active ...
2019/12/10 21:27:17 Waiting for index to go active ...
2019/12/10 21:27:18 Waiting for index to go active ...
2019/12/10 21:27:19 Waiting for index to go active ...
2019/12/10 21:27:20 Waiting for index to go active ...
2019/12/10 21:27:21 Waiting for index to go active ...
2019/12/10 21:27:22 Waiting for index to go active ...
2019/12/10 21:27:23 Waiting for index to go active ...
2019/12/10 21:27:24 Waiting for index to go active ...
2019/12/10 21:27:25 Waiting for index to go active ...
2019/12/10 21:27:26 Waiting for index to go active ...
2019/12/10 21:27:27 Waiting for index to go active ...
2019/12/10 21:27:28 Waiting for index to go active ...
2019/12/10 21:27:29 Waiting for index to go active ...
2019/12/10 21:27:30 Waiting for index to go active ...
2019/12/10 21:27:31 Waiting for index to go active ...
2019/12/10 21:27:32 Waiting for index to go active ...
2019/12/10 21:27:33 Waiting for index to go active ...
2019/12/10 21:27:34 Waiting for index to go active ...
2019/12/10 21:27:35 Waiting for index to go active ...
2019/12/10 21:27:36 Waiting for index to go active ...
2019/12/10 21:27:37 Waiting for index to go active ...
2019/12/10 21:27:38 Waiting for index to go active ...
2019/12/10 21:27:39 Waiting for index to go active ...
2019/12/10 21:27:40 Waiting for index to go active ...
2019/12/10 21:27:41 Waiting for index to go active ...
2019/12/10 21:27:42 Waiting for index to go active ...
2019/12/10 21:27:43 Waiting for index to go active ...
2019/12/10 21:27:44 Waiting for index to go active ...
2019/12/10 21:27:45 Waiting for index to go active ...
2019/12/10 21:27:46 Waiting for index to go active ...
2019/12/10 21:27:47 Waiting for index to go active ...
2019/12/10 21:27:48 Waiting for index to go active ...
2019/12/10 21:27:49 Waiting for index to go active ...
2019/12/10 21:27:50 Waiting for index to go active ...
2019/12/10 21:27:51 Waiting for index to go active ...
2019/12/10 21:27:52 Waiting for index to go active ...
2019/12/10 21:27:53 Waiting for index to go active ...
2019/12/10 21:27:54 Waiting for index to go active ...
2019/12/10 21:27:55 Waiting for index to go active ...
2019/12/10 21:27:56 Waiting for index to go active ...
2019/12/10 21:27:57 Waiting for index to go active ...
2019/12/10 21:27:58 Waiting for index to go active ...
2019/12/10 21:27:59 Waiting for index to go active ...
2019/12/10 21:28:00 Waiting for index to go active ...
2019/12/10 21:28:01 Waiting for index to go active ...
2019/12/10 21:28:02 Waiting for index to go active ...
2019/12/10 21:28:03 Waiting for index to go active ...
2019/12/10 21:28:04 Waiting for index to go active ...
2019/12/10 21:28:05 Waiting for index to go active ...
2019/12/10 21:28:06 Waiting for index to go active ...
2019/12/10 21:28:07 Waiting for index to go active ...
2019/12/10 21:28:08 Waiting for index to go active ...
2019/12/10 21:28:09 Waiting for index to go active ...
2019/12/10 21:28:10 Waiting for index to go active ...
2019/12/10 21:28:11 Waiting for index to go active ...
2019/12/10 21:28:12 Waiting for index to go active ...
2019/12/10 21:28:13 Waiting for index to go active ...
2019/12/10 21:28:14 Waiting for index to go active ...
2019/12/10 21:28:15 Waiting for index to go active ...
2019/12/10 21:28:16 Waiting for index to go active ...
2019/12/10 21:28:17 Waiting for index to go active ...
2019/12/10 21:28:18 Waiting for index to go active ...
2019/12/10 21:28:19 Waiting for index to go active ...
2019/12/10 21:28:20 Waiting for index to go active ...
2019/12/10 21:28:21 Waiting for index to go active ...
2019/12/10 21:28:22 Waiting for index to go active ...
2019/12/10 21:28:23 Waiting for index to go active ...
2019/12/10 21:28:24 Waiting for index to go active ...
2019/12/10 21:28:25 Waiting for index to go active ...
2019/12/10 21:28:26 Waiting for index to go active ...
2019/12/10 21:28:27 Waiting for index to go active ...
2019/12/10 21:28:28 Waiting for index to go active ...
2019/12/10 21:28:29 Waiting for index to go active ...
2019/12/10 21:28:30 Waiting for index to go active ...
2019/12/10 21:28:31 Index is now active
2019/12/10 21:28:32 Using n1ql client
2019/12/10 21:28:33 Expected and Actual scan responses are the same
2019/12/10 21:28:33 Using n1ql client
2019/12/10 21:28:34 Expected and Actual scan responses are the same
--- PASS: TestBuildDeferredAnotherBuilding (394.78s)
=== RUN   TestMultipleBucketsDeferredBuild
2019/12/10 21:28:34 In TestMultipleBucketsDeferredBuild()
2019/12/10 21:28:34 In DropAllSecondaryIndexes()
2019/12/10 21:28:34 Index found:  id_company
2019/12/10 21:28:35 Dropped index id_company
2019/12/10 21:28:35 Index found:  id_age
2019/12/10 21:28:35 Dropped index id_age
2019/12/10 21:28:35 Index found:  id_age1
2019/12/10 21:28:35 Dropped index id_age1
2019/12/10 21:29:43 Flushed the bucket default
2019/12/10 21:29:46 Modified parameters of bucket default
2019/12/10 21:29:46 http://127.0.0.1:9000/pools/default/buckets/defertest_buck2
2019/12/10 21:29:46 &{DELETE http://127.0.0.1:9000/pools/default/buckets/defertest_buck2 HTTP/1.1 1 1 map[Authorization:[Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=] Content-Type:[application/x-www-form-urlencoded; charset=UTF-8]]   0 [] false 127.0.0.1:9000 map[] map[]  map[]      }
2019/12/10 21:29:46 &{404 Object Not Found 404 HTTP/1.1 1 1 map[Expires:[Thu, 01 Jan 1970 00:00:00 GMT] Date:[Tue, 10 Dec 2019 15:59:46 GMT] Content-Length:[31] X-Xss-Protection:[1; mode=block] X-Content-Type-Options:[nosniff] Server:[Couchbase Server] Pragma:[no-cache] X-Permitted-Cross-Domain-Policies:[none] X-Frame-Options:[DENY] Content-Type:[text/plain] Cache-Control:[no-cache,no-store,must-revalidate]] 0xc01f2711c0 31 [] false false map[] 0xc020150e00 }
2019/12/10 21:29:46 DeleteBucket failed for bucket defertest_buck2 
2019/12/10 21:29:46 Deleted bucket defertest_buck2
2019/12/10 21:29:46 Created bucket defertest_buck2
2019/12/10 21:30:16 Setting JSON docs in KV
2019/12/10 21:31:48 Created the secondary index buck1_id1
2019/12/10 21:31:49 Created the secondary index buck1_id2
2019/12/10 21:31:50 Created the secondary index buck2_id3
2019/12/10 21:31:50 Build command issued for the deferred indexes [12990064628638063788]
2019/12/10 21:31:51 Build command issued for the deferred indexes [1267931985035350802 9282902812737092865]
2019/12/10 21:31:51 Index state of 9282902812737092865 is INDEX_STATE_READY
2019/12/10 21:31:51 Waiting for index to go active ...
2019/12/10 21:31:52 Waiting for index to go active ...
2019/12/10 21:31:53 Waiting for index to go active ...
2019/12/10 21:31:54 Waiting for index to go active ...
2019/12/10 21:31:55 Waiting for index to go active ...
2019/12/10 21:31:56 Waiting for index to go active ...
2019/12/10 21:31:57 Waiting for index to go active ...
2019/12/10 21:31:58 Waiting for index to go active ...
2019/12/10 21:31:59 Waiting for index to go active ...
2019/12/10 21:32:00 Waiting for index to go active ...
2019/12/10 21:32:01 Waiting for index to go active ...
2019/12/10 21:32:02 Waiting for index to go active ...
2019/12/10 21:32:03 Waiting for index to go active ...
2019/12/10 21:32:04 Waiting for index to go active ...
2019/12/10 21:32:05 Index is now active
2019/12/10 21:32:05 Waiting for index to go active ...
2019/12/10 21:32:06 Waiting for index to go active ...
2019/12/10 21:32:07 Waiting for index to go active ...
2019/12/10 21:32:08 Waiting for index to go active ...
2019/12/10 21:32:09 Waiting for index to go active ...
2019/12/10 21:32:10 Waiting for index to go active ...
2019/12/10 21:32:11 Waiting for index to go active ...
2019/12/10 21:32:12 Waiting for index to go active ...
2019/12/10 21:32:13 Waiting for index to go active ...
2019/12/10 21:32:14 Waiting for index to go active ...
2019/12/10 21:32:15 Waiting for index to go active ...
2019/12/10 21:32:16 Waiting for index to go active ...
2019/12/10 21:32:17 Waiting for index to go active ...
2019/12/10 21:32:18 Waiting for index to go active ...
2019/12/10 21:32:19 Waiting for index to go active ...
2019/12/10 21:32:20 Waiting for index to go active ...
2019/12/10 21:32:21 Waiting for index to go active ...
2019/12/10 21:32:22 Waiting for index to go active ...
2019/12/10 21:32:23 Waiting for index to go active ...
2019/12/10 21:32:24 Waiting for index to go active ...
2019/12/10 21:32:25 Waiting for index to go active ...
2019/12/10 21:32:26 Waiting for index to go active ...
2019/12/10 21:32:27 Waiting for index to go active ...
2019/12/10 21:32:28 Waiting for index to go active ...
2019/12/10 21:32:29 Waiting for index to go active ...
2019/12/10 21:32:30 Waiting for index to go active ...
2019/12/10 21:32:31 Waiting for index to go active ...
2019/12/10 21:32:32 Waiting for index to go active ...
2019/12/10 21:32:33 Waiting for index to go active ...
2019/12/10 21:32:34 Waiting for index to go active ...
2019/12/10 21:32:35 Waiting for index to go active ...
2019/12/10 21:32:36 Waiting for index to go active ...
2019/12/10 21:32:37 Waiting for index to go active ...
2019/12/10 21:32:38 Waiting for index to go active ...
2019/12/10 21:32:39 Waiting for index to go active ...
2019/12/10 21:32:40 Waiting for index to go active ...
2019/12/10 21:32:41 Waiting for index to go active ...
2019/12/10 21:32:42 Waiting for index to go active ...
2019/12/10 21:32:43 Waiting for index to go active ...
2019/12/10 21:32:44 Waiting for index to go active ...
2019/12/10 21:32:45 Waiting for index to go active ...
2019/12/10 21:32:46 Waiting for index to go active ...
2019/12/10 21:32:47 Waiting for index to go active ...
2019/12/10 21:32:48 Waiting for index to go active ...
2019/12/10 21:32:49 Waiting for index to go active ...
2019/12/10 21:32:50 Waiting for index to go active ...
2019/12/10 21:32:51 Waiting for index to go active ...
2019/12/10 21:32:52 Waiting for index to go active ...
2019/12/10 21:32:53 Waiting for index to go active ...
2019/12/10 21:32:54 Waiting for index to go active ...
2019/12/10 21:32:55 Waiting for index to go active ...
2019/12/10 21:32:56 Waiting for index to go active ...
2019/12/10 21:32:57 Waiting for index to go active ...
2019/12/10 21:32:58 Waiting for index to go active ...
2019/12/10 21:32:59 Waiting for index to go active ...
2019/12/10 21:33:00 Waiting for index to go active ...
2019/12/10 21:33:01 Waiting for index to go active ...
2019/12/10 21:33:02 Waiting for index to go active ...
2019/12/10 21:33:03 Waiting for index to go active ...
2019/12/10 21:33:04 Waiting for index to go active ...
2019/12/10 21:33:05 Waiting for index to go active ...
2019/12/10 21:33:06 Waiting for index to go active ...
2019/12/10 21:33:07 Waiting for index to go active ...
2019/12/10 21:33:08 Waiting for index to go active ...
2019/12/10 21:33:09 Waiting for index to go active ...
2019/12/10 21:33:10 Waiting for index to go active ...
2019/12/10 21:33:11 Waiting for index to go active ...
2019/12/10 21:33:12 Waiting for index to go active ...
2019/12/10 21:33:13 Waiting for index to go active ...
2019/12/10 21:33:14 Waiting for index to go active ...
2019/12/10 21:33:15 Waiting for index to go active ...
2019/12/10 21:33:16 Waiting for index to go active ...
2019/12/10 21:33:17 Waiting for index to go active ...
2019/12/10 21:33:18 Waiting for index to go active ...
2019/12/10 21:33:19 Waiting for index to go active ...
2019/12/10 21:33:20 Waiting for index to go active ...
2019/12/10 21:33:21 Waiting for index to go active ...
2019/12/10 21:33:22 Waiting for index to go active ...
2019/12/10 21:33:23 Waiting for index to go active ...
2019/12/10 21:33:24 Waiting for index to go active ...
2019/12/10 21:33:25 Waiting for index to go active ...
2019/12/10 21:33:26 Waiting for index to go active ...
2019/12/10 21:33:27 Waiting for index to go active ...
2019/12/10 21:33:28 Waiting for index to go active ...
2019/12/10 21:33:29 Waiting for index to go active ...
2019/12/10 21:33:30 Waiting for index to go active ...
2019/12/10 21:33:31 Waiting for index to go active ...
2019/12/10 21:33:32 Waiting for index to go active ...
2019/12/10 21:33:33 Waiting for index to go active ...
2019/12/10 21:33:34 Waiting for index to go active ...
2019/12/10 21:33:35 Waiting for index to go active ...
2019/12/10 21:33:36 Waiting for index to go active ...
2019/12/10 21:33:37 Waiting for index to go active ...
2019/12/10 21:33:38 Waiting for index to go active ...
2019/12/10 21:33:39 Waiting for index to go active ...
2019/12/10 21:33:40 Waiting for index to go active ...
2019/12/10 21:33:41 Waiting for index to go active ...
2019/12/10 21:33:42 Waiting for index to go active ...
2019/12/10 21:33:43 Waiting for index to go active ...
2019/12/10 21:33:44 Waiting for index to go active ...
2019/12/10 21:33:45 Waiting for index to go active ...
2019/12/10 21:33:46 Waiting for index to go active ...
2019/12/10 21:33:47 Waiting for index to go active ...
2019/12/10 21:33:48 Waiting for index to go active ...
2019/12/10 21:33:49 Waiting for index to go active ...
2019/12/10 21:33:50 Waiting for index to go active ...
2019/12/10 21:33:51 Waiting for index to go active ...
2019/12/10 21:33:52 Waiting for index to go active ...
2019/12/10 21:33:53 Waiting for index to go active ...
2019/12/10 21:33:54 Waiting for index to go active ...
2019/12/10 21:33:55 Waiting for index to go active ...
2019/12/10 21:33:56 Waiting for index to go active ...
2019/12/10 21:33:57 Waiting for index to go active ...
2019/12/10 21:33:58 Waiting for index to go active ...
2019/12/10 21:33:59 Waiting for index to go active ...
2019/12/10 21:34:00 Waiting for index to go active ...
2019/12/10 21:34:01 Waiting for index to go active ...
2019/12/10 21:34:02 Waiting for index to go active ...
2019/12/10 21:34:03 Waiting for index to go active ...
2019/12/10 21:34:04 Waiting for index to go active ...
2019/12/10 21:34:05 Waiting for index to go active ...
2019/12/10 21:34:06 Waiting for index to go active ...
2019/12/10 21:34:07 Waiting for index to go active ...
2019/12/10 21:34:08 Waiting for index to go active ...
2019/12/10 21:34:09 Waiting for index to go active ...
2019/12/10 21:34:10 Waiting for index to go active ...
2019/12/10 21:34:11 Waiting for index to go active ...
2019/12/10 21:34:12 Waiting for index to go active ...
2019/12/10 21:34:13 Waiting for index to go active ...
2019/12/10 21:34:14 Waiting for index to go active ...
2019/12/10 21:34:15 Index is now active
2019/12/10 21:34:15 Using n1ql client
2019/12/10 21:34:15 Expected and Actual scan responses are the same
2019/12/10 21:34:15 Using n1ql client
2019/12/10 21:34:15 Expected and Actual scan responses are the same
2019/12/10 21:34:15 Using n1ql client
2019-12-10T21:34:15.737+05:30 [Info] GsiClient::UpdateUsecjson: using collatejson as data format between indexer and GsiClient
2019-12-10T21:34:15.737+05:30 [Info] GSIC[default/defertest_buck2-1575993855734154686] started ...
2019/12/10 21:34:16 Expected and Actual scan responses are the same
2019/12/10 21:34:19 Modified parameters of bucket default
2019/12/10 21:34:21 Deleted bucket defertest_buck2
--- PASS: TestMultipleBucketsDeferredBuild (351.62s)
=== RUN   TestCreateDropCreateDeferredIndex
2019/12/10 21:34:26 In TestCreateDropCreateDeferredIndex()
2019/12/10 21:34:26 In DropAllSecondaryIndexes()
2019/12/10 21:34:26 Index found:  buck1_id2
2019/12/10 21:34:26 Dropped index buck1_id2
2019/12/10 21:34:26 Index found:  buck1_id1
2019/12/10 21:34:26 Dropped index buck1_id1
2019/12/10 21:34:29 Setting JSON docs in KV
2019/12/10 21:34:46 Created the secondary index id_company. Waiting for it become active
2019/12/10 21:34:46 Index is now active
2019/12/10 21:34:47 Created the secondary index id_age
2019/12/10 21:34:47 Dropping the secondary index id_age
2019/12/10 21:34:47 Index dropped
2019/12/10 21:34:50 Setting JSON docs in KV
2019/12/10 21:34:59 Using n1ql client
2019/12/10 21:35:00 Expected and Actual scan responses are the same
--- PASS: TestCreateDropCreateDeferredIndex (33.83s)
=== RUN   TestMultipleDeferredIndexes_BuildTogether
2019/12/10 21:35:00 In TestMultipleDeferredIndexes_BuildTogether()
2019/12/10 21:35:00 In DropAllSecondaryIndexes()
2019/12/10 21:35:00 Index found:  id_company
2019/12/10 21:35:00 Dropped index id_company
2019/12/10 21:35:03 Setting JSON docs in KV
2019/12/10 21:35:22 Created the secondary index id_company. Waiting for it become active
2019/12/10 21:35:22 Index is now active
2019/12/10 21:35:23 Created the secondary index id_age
2019/12/10 21:35:24 Created the secondary index id_gender
2019/12/10 21:35:25 Created the secondary index id_isActive
2019/12/10 21:35:25 Build command issued for the deferred indexes [id_age id_gender id_isActive]
2019/12/10 21:35:25 Waiting for the index id_age to become active
2019/12/10 21:35:25 Waiting for index to go active ...
2019/12/10 21:35:26 Waiting for index to go active ...
2019/12/10 21:35:27 Waiting for index to go active ...
2019/12/10 21:35:28 Waiting for index to go active ...
2019/12/10 21:35:29 Waiting for index to go active ...
2019/12/10 21:35:30 Waiting for index to go active ...
2019/12/10 21:35:31 Waiting for index to go active ...
2019/12/10 21:35:32 Waiting for index to go active ...
2019/12/10 21:35:33 Waiting for index to go active ...
2019/12/10 21:35:34 Waiting for index to go active ...
2019/12/10 21:35:35 Waiting for index to go active ...
2019/12/10 21:35:36 Waiting for index to go active ...
2019/12/10 21:35:37 Waiting for index to go active ...
2019/12/10 21:35:38 Waiting for index to go active ...
2019/12/10 21:35:39 Waiting for index to go active ...
2019/12/10 21:35:40 Waiting for index to go active ...
2019/12/10 21:35:41 Waiting for index to go active ...
2019/12/10 21:35:42 Waiting for index to go active ...
2019/12/10 21:35:43 Index is now active
2019/12/10 21:35:43 Waiting for the index id_gender to become active
2019/12/10 21:35:43 Index is now active
2019/12/10 21:35:43 Waiting for the index id_isActive to become active
2019/12/10 21:35:43 Index is now active
2019/12/10 21:35:43 Using n1ql client
2019/12/10 21:35:44 Expected and Actual scan responses are the same
2019/12/10 21:35:47 Setting JSON docs in KV
2019/12/10 21:35:57 Using n1ql client
2019/12/10 21:35:59 Expected and Actual scan responses are the same
2019/12/10 21:35:59 Using n1ql client
2019/12/10 21:36:00 Expected and Actual scan responses are the same
--- PASS: TestMultipleDeferredIndexes_BuildTogether (59.99s)
=== RUN   TestMultipleDeferredIndexes_BuildOneByOne
2019/12/10 21:36:00 In TestMultipleDeferredIndexes_BuildOneByOne()
2019/12/10 21:36:00 In DropAllSecondaryIndexes()
2019/12/10 21:36:00 Index found:  id_gender
2019/12/10 21:36:00 Dropped index id_gender
2019/12/10 21:36:00 Index found:  id_isActive
2019/12/10 21:36:00 Dropped index id_isActive
2019/12/10 21:36:00 Index found:  id_age
2019/12/10 21:36:01 Dropped index id_age
2019/12/10 21:36:01 Index found:  id_company
2019/12/10 21:36:01 Dropped index id_company
2019/12/10 21:36:04 Setting JSON docs in KV
2019/12/10 21:36:26 Created the secondary index id_company. Waiting for it become active
2019/12/10 21:36:26 Index is now active
2019/12/10 21:36:27 Created the secondary index id_age
2019/12/10 21:36:28 Created the secondary index id_gender
2019/12/10 21:36:29 Created the secondary index id_isActive
2019/12/10 21:36:29 Build command issued for the deferred indexes [id_age]
2019/12/10 21:36:29 Waiting for the index id_age to become active
2019/12/10 21:36:29 Waiting for index to go active ...
2019/12/10 21:36:30 Waiting for index to go active ...
2019/12/10 21:36:31 Waiting for index to go active ...
2019/12/10 21:36:32 Waiting for index to go active ...
2019/12/10 21:36:33 Waiting for index to go active ...
2019/12/10 21:36:34 Waiting for index to go active ...
2019/12/10 21:36:35 Waiting for index to go active ...
2019/12/10 21:36:36 Waiting for index to go active ...
2019/12/10 21:36:37 Waiting for index to go active ...
2019/12/10 21:36:38 Waiting for index to go active ...
2019/12/10 21:36:39 Waiting for index to go active ...
2019/12/10 21:36:40 Waiting for index to go active ...
2019/12/10 21:36:41 Waiting for index to go active ...
2019/12/10 21:36:42 Waiting for index to go active ...
2019/12/10 21:36:43 Index is now active
2019/12/10 21:36:43 Build command issued for the deferred indexes [id_gender]
2019/12/10 21:36:43 Waiting for the index id_gender to become active
2019/12/10 21:36:43 Waiting for index to go active ...
2019/12/10 21:36:44 Waiting for index to go active ...
2019/12/10 21:36:45 Waiting for index to go active ...
2019/12/10 21:36:46 Waiting for index to go active ...
2019/12/10 21:36:47 Waiting for index to go active ...
2019/12/10 21:36:48 Waiting for index to go active ...
2019/12/10 21:36:49 Waiting for index to go active ...
2019/12/10 21:36:50 Waiting for index to go active ...
2019/12/10 21:36:51 Waiting for index to go active ...
2019/12/10 21:36:52 Waiting for index to go active ...
2019/12/10 21:36:53 Waiting for index to go active ...
2019/12/10 21:36:54 Waiting for index to go active ...
2019/12/10 21:36:55 Waiting for index to go active ...
2019/12/10 21:36:56 Waiting for index to go active ...
2019/12/10 21:36:57 Index is now active
2019/12/10 21:36:57 Build command issued for the deferred indexes [id_isActive]
2019/12/10 21:36:57 Waiting for the index id_isActive to become active
2019/12/10 21:36:57 Waiting for index to go active ...
2019/12/10 21:36:58 Waiting for index to go active ...
2019/12/10 21:36:59 Waiting for index to go active ...
2019/12/10 21:37:00 Waiting for index to go active ...
2019/12/10 21:37:01 Waiting for index to go active ...
2019/12/10 21:37:02 Waiting for index to go active ...
2019/12/10 21:37:03 Waiting for index to go active ...
2019/12/10 21:37:04 Waiting for index to go active ...
2019/12/10 21:37:05 Waiting for index to go active ...
2019/12/10 21:37:06 Waiting for index to go active ...
2019/12/10 21:37:07 Waiting for index to go active ...
2019/12/10 21:37:08 Waiting for index to go active ...
2019/12/10 21:37:09 Waiting for index to go active ...
2019/12/10 21:37:10 Waiting for index to go active ...
2019/12/10 21:37:11 Index is now active
2019/12/10 21:37:11 Using n1ql client
2019/12/10 21:37:12 Expected and Actual scan responses are the same
2019/12/10 21:37:14 Setting JSON docs in KV
2019/12/10 21:37:24 Using n1ql client
2019/12/10 21:37:26 Expected and Actual scan responses are the same
2019/12/10 21:37:26 Using n1ql client
2019/12/10 21:37:28 Expected and Actual scan responses are the same
--- PASS: TestMultipleDeferredIndexes_BuildOneByOne (88.04s)
=== RUN   TestDropDeferredIndexWhileOthersBuilding
2019/12/10 21:37:28 In TestDropDeferredIndexWhileOthersBuilding()
2019/12/10 21:37:28 In DropAllSecondaryIndexes()
2019/12/10 21:37:28 Index found:  id_company
2019/12/10 21:37:28 Dropped index id_company
2019/12/10 21:37:28 Index found:  id_age
2019/12/10 21:37:28 Dropped index id_age
2019/12/10 21:37:28 Index found:  id_gender
2019/12/10 21:37:28 Dropped index id_gender
2019/12/10 21:37:28 Index found:  id_isActive
2019/12/10 21:37:29 Dropped index id_isActive
2019/12/10 21:37:32 Setting JSON docs in KV
2019/12/10 21:37:56 Created the secondary index id_company. Waiting for it become active
2019/12/10 21:37:56 Index is now active
2019/12/10 21:37:58 Created the secondary index id_age
2019/12/10 21:37:59 Created the secondary index id_gender
2019/12/10 21:38:00 Created the secondary index id_isActive
2019/12/10 21:38:00 Build command issued for the deferred indexes [17685127763204542216 1173848903035885780]
2019/12/10 21:38:02 Dropping the secondary index id_isActive
2019/12/10 21:38:02 Index dropped
2019/12/10 21:38:02 Waiting for index to go active ...
2019/12/10 21:38:03 Waiting for index to go active ...
2019/12/10 21:38:04 Waiting for index to go active ...
2019/12/10 21:38:05 Waiting for index to go active ...
2019/12/10 21:38:06 Waiting for index to go active ...
2019/12/10 21:38:07 Waiting for index to go active ...
2019/12/10 21:38:08 Waiting for index to go active ...
2019/12/10 21:38:09 Waiting for index to go active ...
2019/12/10 21:38:10 Waiting for index to go active ...
2019/12/10 21:38:11 Waiting for index to go active ...
2019/12/10 21:38:12 Waiting for index to go active ...
2019/12/10 21:38:13 Waiting for index to go active ...
2019/12/10 21:38:14 Waiting for index to go active ...
2019/12/10 21:38:15 Waiting for index to go active ...
2019/12/10 21:38:16 Waiting for index to go active ...
2019/12/10 21:38:17 Waiting for index to go active ...
2019/12/10 21:38:18 Waiting for index to go active ...
2019/12/10 21:38:19 Index is now active
2019/12/10 21:38:19 Index is now active
2019/12/10 21:38:20 Using n1ql client
2019/12/10 21:38:21 Expected and Actual scan responses are the same
2019/12/10 21:38:21 Using n1ql client
2019/12/10 21:38:21 Expected and Actual scan responses are the same
2019/12/10 21:38:23 Setting JSON docs in KV
2019/12/10 21:38:34 Using n1ql client
2019/12/10 21:38:36 Expected and Actual scan responses are the same
--- PASS: TestDropDeferredIndexWhileOthersBuilding (67.77s)
=== RUN   TestDropBuildingDeferredIndex
2019/12/10 21:38:36 In TestDropBuildingDeferredIndex()
2019/12/10 21:38:36 In DropAllSecondaryIndexes()
2019/12/10 21:38:36 Index found:  id_gender
2019/12/10 21:38:36 Dropped index id_gender
2019/12/10 21:38:36 Index found:  id_age
2019/12/10 21:38:36 Dropped index id_age
2019/12/10 21:38:36 Index found:  id_company
2019/12/10 21:38:36 Dropped index id_company
2019/12/10 21:38:39 Setting JSON docs in KV
2019/12/10 21:38:46 Created the secondary index id_company
2019/12/10 21:38:47 Created the secondary index id_age
2019/12/10 21:38:48 Created the secondary index id_gender
2019/12/10 21:38:48 Build command issued for the deferred indexes [18332824135871315249 4058307916558412023]
2019/12/10 21:38:49 Dropping the secondary index id_age
2019/12/10 21:38:50 Index dropped
2019/12/10 21:38:50 Waiting for index to go active ...
2019/12/10 21:38:51 Waiting for index to go active ...
2019/12/10 21:38:52 Waiting for index to go active ...
2019/12/10 21:38:53 Waiting for index to go active ...
2019/12/10 21:38:54 Waiting for index to go active ...
2019/12/10 21:38:55 Waiting for index to go active ...
2019/12/10 21:38:56 Waiting for index to go active ...
2019/12/10 21:38:57 Waiting for index to go active ...
2019/12/10 21:38:58 Waiting for index to go active ...
2019/12/10 21:38:59 Waiting for index to go active ...
2019/12/10 21:39:00 Waiting for index to go active ...
2019/12/10 21:39:01 Waiting for index to go active ...
2019/12/10 21:39:02 Waiting for index to go active ...
2019/12/10 21:39:03 Waiting for index to go active ...
2019/12/10 21:39:04 Waiting for index to go active ...
2019/12/10 21:39:05 Waiting for index to go active ...
2019/12/10 21:39:06 Waiting for index to go active ...
2019/12/10 21:39:07 Waiting for index to go active ...
2019/12/10 21:39:08 Waiting for index to go active ...
2019/12/10 21:39:09 Waiting for index to go active ...
2019/12/10 21:39:10 Index is now active
2019/12/10 21:39:10 Build command issued for the deferred indexes [id_gender]
2019/12/10 21:39:10 Waiting for the index id_gender to become active
2019/12/10 21:39:10 Waiting for index to go active ...
2019/12/10 21:39:11 Waiting for index to go active ...
2019/12/10 21:39:12 Waiting for index to go active ...
2019/12/10 21:39:13 Waiting for index to go active ...
2019/12/10 21:39:14 Waiting for index to go active ...
2019/12/10 21:39:15 Waiting for index to go active ...
2019/12/10 21:39:16 Waiting for index to go active ...
2019/12/10 21:39:17 Waiting for index to go active ...
2019/12/10 21:39:18 Waiting for index to go active ...
2019/12/10 21:39:19 Waiting for index to go active ...
2019/12/10 21:39:20 Waiting for index to go active ...
2019/12/10 21:39:21 Waiting for index to go active ...
2019/12/10 21:39:22 Waiting for index to go active ...
2019/12/10 21:39:23 Waiting for index to go active ...
2019/12/10 21:39:24 Waiting for index to go active ...
2019/12/10 21:39:25 Waiting for index to go active ...
2019/12/10 21:39:26 Waiting for index to go active ...
2019/12/10 21:39:27 Waiting for index to go active ...
2019/12/10 21:39:28 Waiting for index to go active ...
2019/12/10 21:39:29 Index is now active
2019/12/10 21:39:30 Using n1ql client
2019/12/10 21:39:31 Expected and Actual scan responses are the same
2019/12/10 21:39:31 Using n1ql client
2019/12/10 21:39:32 Expected and Actual scan responses are the same
2019/12/10 21:39:35 Setting JSON docs in KV
2019/12/10 21:39:44 Using n1ql client
2019/12/10 21:39:49 Expected and Actual scan responses are the same
--- PASS: TestDropBuildingDeferredIndex (73.08s)
=== RUN   TestDropMultipleBuildingDeferredIndexes
2019/12/10 21:39:49 In TestDropMultipleBuildingDeferredIndexes()
2019/12/10 21:39:49 In DropAllSecondaryIndexes()
2019/12/10 21:39:49 Index found:  id_company
2019/12/10 21:39:49 Dropped index id_company
2019/12/10 21:39:49 Index found:  id_gender
2019/12/10 21:39:49 Dropped index id_gender
2019/12/10 21:39:57 Setting JSON docs in KV
2019/12/10 21:40:42 Created the secondary index id_company. Waiting for it become active
2019/12/10 21:40:42 Index is now active
2019/12/10 21:40:44 Created the secondary index id_age
2019/12/10 21:40:44 Created the secondary index id_gender
2019/12/10 21:40:45 Created the secondary index id_isActive
2019/12/10 21:40:46 Build command issued for the deferred indexes [4809295137930123851 10488967299743865747]
2019/12/10 21:40:47 Dropping the secondary index id_age
2019/12/10 21:40:47 Index dropped
2019/12/10 21:40:47 Dropping the secondary index id_gender
2019/12/10 21:41:04 Index dropped
2019/12/10 21:41:05 Build command issued for the deferred indexes [id_isActive]
2019/12/10 21:41:05 Waiting for the index id_isActive to become active
2019/12/10 21:41:05 Waiting for index to go active ...
2019/12/10 21:41:06 Waiting for index to go active ...
2019/12/10 21:41:07 Waiting for index to go active ...
2019/12/10 21:41:08 Waiting for index to go active ...
2019/12/10 21:41:09 Waiting for index to go active ...
2019/12/10 21:41:10 Waiting for index to go active ...
2019/12/10 21:41:11 Waiting for index to go active ...
2019/12/10 21:41:12 Waiting for index to go active ...
2019/12/10 21:41:13 Waiting for index to go active ...
2019/12/10 21:41:14 Waiting for index to go active ...
2019/12/10 21:41:15 Waiting for index to go active ...
2019/12/10 21:41:16 Waiting for index to go active ...
2019/12/10 21:41:17 Waiting for index to go active ...
2019/12/10 21:41:18 Waiting for index to go active ...
2019/12/10 21:41:19 Waiting for index to go active ...
2019/12/10 21:41:20 Waiting for index to go active ...
2019/12/10 21:41:21 Waiting for index to go active ...
2019/12/10 21:41:22 Waiting for index to go active ...
2019/12/10 21:41:23 Waiting for index to go active ...
2019/12/10 21:41:24 Waiting for index to go active ...
2019/12/10 21:41:25 Waiting for index to go active ...
2019/12/10 21:41:26 Waiting for index to go active ...
2019/12/10 21:41:27 Waiting for index to go active ...
2019/12/10 21:41:28 Waiting for index to go active ...
2019/12/10 21:41:29 Index is now active
2019/12/10 21:41:39 Using n1ql client
2019/12/10 21:41:42 Expected and Actual scan responses are the same
2019/12/10 21:41:42 Number of docScanResults and scanResults = 180000 and 180000
2019/12/10 21:41:42 Using n1ql client
2019/12/10 21:41:47 Expected and Actual scan responses are the same
2019/12/10 21:41:47 Number of docScanResults and scanResults = 180000 and 180000
--- PASS: TestDropMultipleBuildingDeferredIndexes (118.02s)
=== RUN   TestDropOneIndexSecondDeferBuilding
2019/12/10 21:41:47 In TestDropOneIndexSecondDeferBuilding()
2019/12/10 21:41:47 In DropAllSecondaryIndexes()
2019/12/10 21:41:47 Index found:  id_company
2019/12/10 21:41:47 Dropped index id_company
2019/12/10 21:41:47 Index found:  id_isActive
2019/12/10 21:41:47 Dropped index id_isActive
2019/12/10 21:41:49 Setting JSON docs in KV
2019/12/10 21:41:56 Created the secondary index id_company
2019/12/10 21:41:57 Created the secondary index id_age
2019/12/10 21:41:58 Created the secondary index id_gender
2019/12/10 21:41:59 Build command issued for the deferred indexes [id_company]
2019/12/10 21:41:59 Waiting for the index id_company to become active
2019/12/10 21:41:59 Waiting for index to go active ...
2019/12/10 21:42:00 Waiting for index to go active ...
2019/12/10 21:42:01 Waiting for index to go active ...
2019/12/10 21:42:02 Waiting for index to go active ...
2019/12/10 21:42:03 Waiting for index to go active ...
2019/12/10 21:42:04 Waiting for index to go active ...
2019/12/10 21:42:05 Waiting for index to go active ...
2019/12/10 21:42:06 Waiting for index to go active ...
2019/12/10 21:42:07 Waiting for index to go active ...
2019/12/10 21:42:08 Waiting for index to go active ...
2019/12/10 21:42:09 Waiting for index to go active ...
2019/12/10 21:42:10 Waiting for index to go active ...
2019/12/10 21:42:11 Waiting for index to go active ...
2019/12/10 21:42:12 Waiting for index to go active ...
2019/12/10 21:42:13 Waiting for index to go active ...
2019/12/10 21:42:14 Waiting for index to go active ...
2019/12/10 21:42:15 Waiting for index to go active ...
2019/12/10 21:42:16 Waiting for index to go active ...
2019/12/10 21:42:17 Waiting for index to go active ...
2019/12/10 21:42:18 Waiting for index to go active ...
2019/12/10 21:42:19 Waiting for index to go active ...
2019/12/10 21:42:20 Waiting for index to go active ...
2019/12/10 21:42:21 Waiting for index to go active ...
2019/12/10 21:42:22 Waiting for index to go active ...
2019/12/10 21:42:23 Index is now active
2019/12/10 21:42:23 Build command issued for the deferred indexes [18280836784186664584]
2019/12/10 21:42:24 Dropping the secondary index id_company
2019/12/10 21:42:24 Index dropped
2019/12/10 21:42:34 Setting JSON docs in KV
2019/12/10 21:42:51 Waiting for index to go active ...
2019/12/10 21:42:52 Waiting for index to go active ...
2019/12/10 21:42:53 Waiting for index to go active ...
2019/12/10 21:42:54 Waiting for index to go active ...
2019/12/10 21:42:55 Waiting for index to go active ...
2019/12/10 21:42:56 Index is now active
2019/12/10 21:42:56 Build command issued for the deferred indexes [id_gender]
2019/12/10 21:42:56 Waiting for the index id_gender to become active
2019/12/10 21:42:56 Waiting for index to go active ...
2019/12/10 21:42:57 Waiting for index to go active ...
2019/12/10 21:42:58 Waiting for index to go active ...
2019/12/10 21:42:59 Waiting for index to go active ...
2019/12/10 21:43:00 Waiting for index to go active ...
2019/12/10 21:43:01 Waiting for index to go active ...
2019/12/10 21:43:02 Waiting for index to go active ...
2019/12/10 21:43:03 Waiting for index to go active ...
2019/12/10 21:43:04 Waiting for index to go active ...
2019/12/10 21:43:05 Waiting for index to go active ...
2019/12/10 21:43:06 Waiting for index to go active ...
2019/12/10 21:43:07 Waiting for index to go active ...
2019/12/10 21:43:08 Waiting for index to go active ...
2019/12/10 21:43:09 Waiting for index to go active ...
2019/12/10 21:43:10 Waiting for index to go active ...
2019/12/10 21:43:11 Waiting for index to go active ...
2019/12/10 21:43:12 Waiting for index to go active ...
2019/12/10 21:43:13 Waiting for index to go active ...
2019/12/10 21:43:14 Waiting for index to go active ...
2019/12/10 21:43:15 Waiting for index to go active ...
2019/12/10 21:43:16 Waiting for index to go active ...
2019/12/10 21:43:17 Waiting for index to go active ...
2019/12/10 21:43:18 Waiting for index to go active ...
2019/12/10 21:43:19 Waiting for index to go active ...
2019/12/10 21:43:20 Waiting for index to go active ...
2019/12/10 21:43:21 Index is now active
2019/12/10 21:43:21 Using n1ql client
2019/12/10 21:43:22 Expected and Actual scan responses are the same
2019/12/10 21:43:22 Using n1ql client
2019/12/10 21:43:26 Expected and Actual scan responses are the same
--- PASS: TestDropOneIndexSecondDeferBuilding (98.98s)
=== RUN   TestDropSecondIndexSecondDeferBuilding
2019/12/10 21:43:26 In TestDropSecondIndexSecondDeferBuilding()
2019/12/10 21:43:26 In DropAllSecondaryIndexes()
2019/12/10 21:43:26 Index found:  id_gender
2019/12/10 21:43:26 Dropped index id_gender
2019/12/10 21:43:26 Index found:  id_age
2019/12/10 21:43:26 Dropped index id_age
2019/12/10 21:43:29 Setting JSON docs in KV
2019/12/10 21:43:36 Created the secondary index id_company
2019/12/10 21:43:37 Created the secondary index id_age
2019/12/10 21:43:38 Created the secondary index id_gender
2019/12/10 21:43:38 Build command issued for the deferred indexes [id_company]
2019/12/10 21:43:38 Waiting for the index id_company to become active
2019/12/10 21:43:38 Waiting for index to go active ...
2019/12/10 21:43:39 Waiting for index to go active ...
2019/12/10 21:43:40 Waiting for index to go active ...
2019/12/10 21:43:41 Waiting for index to go active ...
2019/12/10 21:43:42 Waiting for index to go active ...
2019/12/10 21:43:43 Waiting for index to go active ...
2019/12/10 21:43:44 Waiting for index to go active ...
2019/12/10 21:43:45 Waiting for index to go active ...
2019/12/10 21:43:46 Waiting for index to go active ...
2019/12/10 21:43:47 Waiting for index to go active ...
2019/12/10 21:43:48 Waiting for index to go active ...
2019/12/10 21:43:49 Waiting for index to go active ...
2019/12/10 21:43:50 Waiting for index to go active ...
2019/12/10 21:43:51 Waiting for index to go active ...
2019/12/10 21:43:52 Waiting for index to go active ...
2019/12/10 21:43:53 Waiting for index to go active ...
2019/12/10 21:43:54 Waiting for index to go active ...
2019/12/10 21:43:55 Waiting for index to go active ...
2019/12/10 21:43:56 Waiting for index to go active ...
2019/12/10 21:43:57 Waiting for index to go active ...
2019/12/10 21:43:58 Waiting for index to go active ...
2019/12/10 21:43:59 Waiting for index to go active ...
2019/12/10 21:44:00 Waiting for index to go active ...
2019/12/10 21:44:01 Waiting for index to go active ...
2019/12/10 21:44:02 Waiting for index to go active ...
2019/12/10 21:44:03 Waiting for index to go active ...
2019/12/10 21:44:04 Waiting for index to go active ...
2019/12/10 21:44:05 Waiting for index to go active ...
2019/12/10 21:44:06 Index is now active
2019/12/10 21:44:07 Build command issued for the deferred indexes [9949814246020900605]
2019/12/10 21:44:08 Dropping the secondary index id_age
2019/12/10 21:44:08 Index dropped
2019/12/10 21:44:14 Setting JSON docs in KV
2019/12/10 21:44:24 Build command issued for the deferred indexes [id_gender]
2019/12/10 21:44:24 Waiting for the index id_gender to become active
2019/12/10 21:44:24 Waiting for index to go active ...
2019/12/10 21:44:25 Waiting for index to go active ...
2019/12/10 21:44:26 Waiting for index to go active ...
2019/12/10 21:44:27 Waiting for index to go active ...
2019/12/10 21:44:28 Waiting for index to go active ...
2019/12/10 21:44:29 Waiting for index to go active ...
2019/12/10 21:44:30 Waiting for index to go active ...
2019/12/10 21:44:31 Waiting for index to go active ...
2019/12/10 21:44:32 Waiting for index to go active ...
2019/12/10 21:44:33 Waiting for index to go active ...
2019/12/10 21:44:34 Waiting for index to go active ...
2019/12/10 21:44:35 Waiting for index to go active ...
2019/12/10 21:44:36 Waiting for index to go active ...
2019/12/10 21:44:37 Waiting for index to go active ...
2019/12/10 21:44:38 Waiting for index to go active ...
2019/12/10 21:44:39 Waiting for index to go active ...
2019/12/10 21:44:40 Waiting for index to go active ...
2019/12/10 21:44:41 Waiting for index to go active ...
2019/12/10 21:44:42 Waiting for index to go active ...
2019/12/10 21:44:43 Waiting for index to go active ...
2019/12/10 21:44:44 Waiting for index to go active ...
2019/12/10 21:44:45 Waiting for index to go active ...
2019/12/10 21:44:46 Waiting for index to go active ...
2019/12/10 21:44:47 Waiting for index to go active ...
2019/12/10 21:44:48 Waiting for index to go active ...
2019/12/10 21:44:49 Waiting for index to go active ...
2019/12/10 21:44:50 Waiting for index to go active ...
2019/12/10 21:44:51 Waiting for index to go active ...
2019/12/10 21:44:52 Index is now active
2019/12/10 21:44:52 Using n1ql client
2019/12/10 21:44:53 Expected and Actual scan responses are the same
2019/12/10 21:44:53 Using n1ql client
2019/12/10 21:44:58 Expected and Actual scan responses are the same
--- PASS: TestDropSecondIndexSecondDeferBuilding (92.19s)
=== RUN   TestCreateAfterDropWhileIndexBuilding
2019/12/10 21:44:58 In TestCreateAfterDropWhileIndexBuilding()
2019/12/10 21:44:58 In DropAllSecondaryIndexes()
2019/12/10 21:44:58 Index found:  id_gender
2019/12/10 21:44:58 Dropped index id_gender
2019/12/10 21:44:58 Index found:  id_company
2019/12/10 21:44:58 Dropped index id_company
2019/12/10 21:45:24 Setting JSON docs in KV
2019/12/10 21:46:36 Created the secondary index id_company
2019/12/10 21:46:36 Build command issued for the deferred indexes [18250850008215274188]
2019/12/10 21:46:37 Waiting for index to go active ...
2019/12/10 21:46:38 Waiting for index to go active ...
2019/12/10 21:46:39 Waiting for index to go active ...
2019/12/10 21:46:40 Waiting for index to go active ...
2019/12/10 21:46:41 Waiting for index to go active ...
2019/12/10 21:46:42 Waiting for index to go active ...
2019/12/10 21:46:43 Waiting for index to go active ...
2019/12/10 21:46:44 Waiting for index to go active ...
2019/12/10 21:46:45 Waiting for index to go active ...
2019/12/10 21:46:46 Waiting for index to go active ...
2019/12/10 21:46:47 Waiting for index to go active ...
2019/12/10 21:46:48 Waiting for index to go active ...
2019/12/10 21:46:49 Waiting for index to go active ...
2019/12/10 21:46:50 Waiting for index to go active ...
2019/12/10 21:46:51 Waiting for index to go active ...
2019/12/10 21:46:52 Waiting for index to go active ...
2019/12/10 21:46:53 Waiting for index to go active ...
2019/12/10 21:46:54 Waiting for index to go active ...
2019/12/10 21:46:55 Waiting for index to go active ...
2019/12/10 21:46:56 Waiting for index to go active ...
2019/12/10 21:46:57 Waiting for index to go active ...
2019/12/10 21:46:58 Waiting for index to go active ...
2019/12/10 21:46:59 Waiting for index to go active ...
2019/12/10 21:47:00 Waiting for index to go active ...
2019/12/10 21:47:01 Waiting for index to go active ...
2019/12/10 21:47:02 Waiting for index to go active ...
2019/12/10 21:47:03 Waiting for index to go active ...
2019/12/10 21:47:04 Waiting for index to go active ...
2019/12/10 21:47:05 Waiting for index to go active ...
2019/12/10 21:47:06 Waiting for index to go active ...
2019/12/10 21:47:07 Waiting for index to go active ...
2019/12/10 21:47:08 Waiting for index to go active ...
2019/12/10 21:47:09 Waiting for index to go active ...
2019/12/10 21:47:10 Waiting for index to go active ...
2019/12/10 21:47:11 Waiting for index to go active ...
2019/12/10 21:47:12 Waiting for index to go active ...
2019/12/10 21:47:13 Waiting for index to go active ...
2019/12/10 21:47:14 Waiting for index to go active ...
2019/12/10 21:47:15 Waiting for index to go active ...
2019/12/10 21:47:16 Waiting for index to go active ...
2019/12/10 21:47:17 Index is now active
2019/12/10 21:47:18 Created the secondary index id_age
2019/12/10 21:47:18 Build command issued for the deferred indexes [4075622842208913197]
2019/12/10 21:47:19 Dropping the secondary index id_company
2019/12/10 21:47:20 Index dropped
2019/12/10 21:47:20 Dropping the secondary index id_age
2019/12/10 21:47:20 Index dropped
2019/12/10 21:47:29 Created the secondary index id_gender
2019/12/10 21:47:29 Build command issued for the deferred indexes [id_gender]
2019/12/10 21:47:29 Waiting for the index id_gender to become active
2019/12/10 21:47:29 Waiting for index to go active ...
2019/12/10 21:47:30 Waiting for index to go active ...
2019/12/10 21:47:31 Waiting for index to go active ...
2019/12/10 21:47:32 Waiting for index to go active ...
2019/12/10 21:47:33 Waiting for index to go active ...
2019/12/10 21:47:34 Waiting for index to go active ...
2019/12/10 21:47:35 Waiting for index to go active ...
2019/12/10 21:47:36 Waiting for index to go active ...
2019/12/10 21:47:37 Waiting for index to go active ...
2019/12/10 21:47:38 Waiting for index to go active ...
2019/12/10 21:47:39 Waiting for index to go active ...
2019/12/10 21:47:40 Waiting for index to go active ...
2019/12/10 21:47:41 Waiting for index to go active ...
2019/12/10 21:47:42 Waiting for index to go active ...
2019/12/10 21:47:43 Waiting for index to go active ...
2019/12/10 21:47:44 Waiting for index to go active ...
2019/12/10 21:47:45 Waiting for index to go active ...
2019/12/10 21:47:46 Waiting for index to go active ...
2019/12/10 21:47:47 Waiting for index to go active ...
2019/12/10 21:47:48 Waiting for index to go active ...
2019/12/10 21:47:49 Waiting for index to go active ...
2019/12/10 21:47:50 Waiting for index to go active ...
2019/12/10 21:47:51 Waiting for index to go active ...
2019/12/10 21:47:52 Waiting for index to go active ...
2019/12/10 21:47:53 Waiting for index to go active ...
2019/12/10 21:47:54 Waiting for index to go active ...
2019/12/10 21:47:55 Waiting for index to go active ...
2019/12/10 21:47:56 Waiting for index to go active ...
2019/12/10 21:47:57 Waiting for index to go active ...
2019/12/10 21:47:58 Waiting for index to go active ...
2019/12/10 21:47:59 Waiting for index to go active ...
2019/12/10 21:48:00 Waiting for index to go active ...
2019/12/10 21:48:01 Waiting for index to go active ...
2019/12/10 21:48:02 Waiting for index to go active ...
2019/12/10 21:48:03 Waiting for index to go active ...
2019/12/10 21:48:04 Waiting for index to go active ...
2019/12/10 21:48:05 Waiting for index to go active ...
2019/12/10 21:48:06 Index is now active
2019/12/10 21:48:07 Index is now active
2019/12/10 21:48:07 Using n1ql client
2019/12/10 21:48:11 Expected and Actual scan responses are the same
--- PASS: TestCreateAfterDropWhileIndexBuilding (192.78s)
=== RUN   TestDropBuildingIndex1
2019/12/10 21:48:11 In TestDropBuildingIndex1()
2019/12/10 21:48:11 In DropAllSecondaryIndexes()
2019/12/10 21:48:11 Index found:  id_gender
2019/12/10 21:48:11 Dropped index id_gender
2019/12/10 21:48:15 Setting JSON docs in KV
2019/12/10 21:49:12 Created the secondary index id_company. Waiting for it become active
2019/12/10 21:49:12 Index is now active
2019/12/10 21:49:53 Created the secondary index id_age
2019/12/10 21:49:54 Dropping the secondary index id_age
2019/12/10 21:49:54 Index dropped
2019/12/10 21:50:37 Created the secondary index id_age. Waiting for it become active
2019/12/10 21:50:37 Index is now active
2019/12/10 21:50:39 Setting JSON docs in KV
2019/12/10 21:50:49 Using n1ql client
2019/12/10 21:50:51 Expected and Actual scan responses are the same
2019/12/10 21:50:51 Using n1ql client
2019/12/10 21:50:53 Expected and Actual scan responses are the same
--- PASS: TestDropBuildingIndex1 (161.90s)
=== RUN   TestDropBuildingIndex2
2019/12/10 21:50:53 In TestDropBuildingIndex2()
2019/12/10 21:50:53 In DropAllSecondaryIndexes()
2019/12/10 21:50:53 Index found:  id_company
2019/12/10 21:50:53 Dropped index id_company
2019/12/10 21:50:53 Index found:  id_age
2019/12/10 21:50:53 Dropped index id_age
2019/12/10 21:50:59 Setting JSON docs in KV
2019/12/10 21:52:02 Created the secondary index id_company. Waiting for it become active
2019/12/10 21:52:02 Index is now active
2019/12/10 21:52:47 Created the secondary index id_age
2019/12/10 21:52:48 Dropping the secondary index id_company
2019/12/10 21:52:48 Index dropped
2019/12/10 21:52:48 Index is now active
2019/12/10 21:53:34 Created the secondary index id_company. Waiting for it become active
2019/12/10 21:53:34 Index is now active
2019/12/10 21:53:36 Setting JSON docs in KV
2019/12/10 21:53:46 Using n1ql client
2019/12/10 21:53:48 Expected and Actual scan responses are the same
2019/12/10 21:53:48 Using n1ql client
2019/12/10 21:53:50 Expected and Actual scan responses are the same
--- PASS: TestDropBuildingIndex2 (177.41s)
=== RUN   TestDropIndexWithDataLoad
2019/12/10 21:53:50 In TestDropIndexWithDataLoad()
2019/12/10 21:53:50 In DropAllSecondaryIndexes()
2019/12/10 21:53:50 Index found:  id_age
2019/12/10 21:53:50 Dropped index id_age
2019/12/10 21:53:50 Index found:  id_company
2019/12/10 21:53:51 Dropped index id_company
2019/12/10 21:53:53 Setting JSON docs in KV
2019/12/10 21:54:47 Created the secondary index id_company. Waiting for it become active
2019/12/10 21:54:47 Index is now active
2019/12/10 21:55:36 Created the secondary index id_age. Waiting for it become active
2019/12/10 21:55:36 Index is now active
2019/12/10 21:56:25 Created the secondary index id_gender. Waiting for it become active
2019/12/10 21:56:25 Index is now active
2019/12/10 21:57:16 Created the secondary index id_isActive. Waiting for it become active
2019/12/10 21:57:16 Index is now active
2019/12/10 21:57:23 Setting JSON docs in KV
2019/12/10 21:57:23 In LoadKVBucket
2019/12/10 21:57:23 Bucket name = default
2019/12/10 21:57:23 In DropIndexWhileKVLoad
2019/12/10 21:57:24 Dropping the secondary index id_company
2019/12/10 21:57:25 Index dropped
2019/12/10 21:57:58 Using n1ql client
2019/12/10 21:58:00 Expected and Actual scan responses are the same
2019/12/10 21:58:00 Number of docScanResults and scanResults = 96752 and 96752
2019/12/10 21:58:01 Using n1ql client
2019/12/10 21:58:09 Expected and Actual scan responses are the same
2019/12/10 21:58:09 Number of docScanResults and scanResults = 420000 and 420000
--- PASS: TestDropIndexWithDataLoad (259.28s)
=== RUN   TestDropAllIndexesWithDataLoad
2019/12/10 21:58:09 In TestDropAllIndexesWithDataLoad()
2019/12/10 21:58:09 In DropAllSecondaryIndexes()
2019/12/10 21:58:09 Index found:  id_isActive
2019/12/10 21:58:09 Dropped index id_isActive
2019/12/10 21:58:09 Index found:  id_age
2019/12/10 21:58:10 Dropped index id_age
2019/12/10 21:58:10 Index found:  id_gender
2019/12/10 21:58:10 Dropped index id_gender
2019/12/10 21:58:12 Setting JSON docs in KV
2019/12/10 21:59:16 Created the secondary index id_company. Waiting for it become active
2019/12/10 21:59:16 Index is now active
2019/12/10 22:00:11 Created the secondary index id_age. Waiting for it become active
2019/12/10 22:00:11 Index is now active
2019/12/10 22:01:05 Created the secondary index id_gender. Waiting for it become active
2019/12/10 22:01:05 Index is now active
2019/12/10 22:02:01 Created the secondary index id_isActive. Waiting for it become active
2019/12/10 22:02:01 Index is now active
2019/12/10 22:02:08 Setting JSON docs in KV
2019/12/10 22:02:08 In LoadKVBucket
2019/12/10 22:02:08 Bucket name = default
2019/12/10 22:02:08 In DropIndexWhileKVLoad
2019/12/10 22:02:08 In DropIndexWhileKVLoad
2019/12/10 22:02:08 In DropIndexWhileKVLoad
2019/12/10 22:02:08 In DropIndexWhileKVLoad
2019/12/10 22:02:09 Dropping the secondary index id_isActive
2019/12/10 22:02:09 Dropping the secondary index id_company
2019/12/10 22:02:09 Dropping the secondary index id_gender
2019/12/10 22:02:09 Dropping the secondary index id_age
2019/12/10 22:02:10 Index dropped
2019/12/10 22:02:10 Index dropped
2019/12/10 22:02:10 Index dropped
2019/12/10 22:02:11 Index dropped
2019/12/10 22:02:31 Using n1ql client
2019/12/10 22:02:31 Scan failed as expected with error: GSI index id_company not found.
--- PASS: TestDropAllIndexesWithDataLoad (261.87s)
=== RUN   TestCreateBucket_AnotherIndexBuilding
2019/12/10 22:02:31 In TestCreateBucket_AnotherIndexBuilding()
2019/12/10 22:02:31 In DropAllSecondaryIndexes()
2019/12/10 22:03:39 Flushed the bucket default
2019/12/10 22:03:42 Modified parameters of bucket default
2019/12/10 22:03:42 http://127.0.0.1:9000/pools/default/buckets/multi_buck2
2019/12/10 22:03:42 &{DELETE http://127.0.0.1:9000/pools/default/buckets/multi_buck2 HTTP/1.1 1 1 map[Content-Type:[application/x-www-form-urlencoded; charset=UTF-8] Authorization:[Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=]]   0 [] false 127.0.0.1:9000 map[] map[]  map[]      }
2019/12/10 22:03:42 &{404 Object Not Found 404 HTTP/1.1 1 1 map[X-Frame-Options:[DENY] X-Content-Type-Options:[nosniff] Server:[Couchbase Server] Pragma:[no-cache] Content-Type:[text/plain] Content-Length:[31] X-Xss-Protection:[1; mode=block] X-Permitted-Cross-Domain-Policies:[none] Cache-Control:[no-cache,no-store,must-revalidate] Expires:[Thu, 01 Jan 1970 00:00:00 GMT] Date:[Tue, 10 Dec 2019 16:33:42 GMT]] 0xc0b39e6e40 31 [] false false map[] 0xc0b3313000 }
2019/12/10 22:03:42 DeleteBucket failed for bucket multi_buck2 
2019/12/10 22:03:42 Deleted bucket multi_buck2
2019/12/10 22:04:12 Setting JSON docs in KV
2019/12/10 22:06:46 Created the secondary index buck1_idx
2019/12/10 22:06:46 Created bucket multi_buck2
2019/12/10 22:07:09 Created the secondary index buck2_idx
2019/12/10 22:07:09 Index is now active
2019/12/10 22:07:09 Index is now active
2019/12/10 22:07:09 Using n1ql client
2019-12-10T22:07:09.755+05:30 [Info] metadata provider version changed 1128 -> 1129
2019-12-10T22:07:09.755+05:30 [Info] switched currmeta from 1128 -> 1129 force false 
2019-12-10T22:07:09.755+05:30 [Info] GsiClient::UpdateUsecjson: using collatejson as data format between indexer and GsiClient
2019-12-10T22:07:09.755+05:30 [Info] GSIC[default/multi_buck2-1575995829751269251] started ...
2019/12/10 22:07:09 Expected and Actual scan responses are the same
2019/12/10 22:07:09 Number of docScanResults and scanResults = 10000 and 10000
2019/12/10 22:07:10 Using n1ql client
2019/12/10 22:07:14 Expected and Actual scan responses are the same
2019/12/10 22:07:14 Number of docScanResults and scanResults = 200000 and 200000
2019/12/10 22:07:16 Deleted bucket multi_buck2
2019/12/10 22:08:24 Flushed the bucket default
--- PASS: TestCreateBucket_AnotherIndexBuilding (352.98s)
=== RUN   TestDropBucket2Index_Bucket1IndexBuilding
2019/12/10 22:08:24 In TestDropBucket2Index_Bucket1IndexBuilding()
2019/12/10 22:08:24 In DropAllSecondaryIndexes()
2019/12/10 22:08:24 Index found:  buck1_idx
2019/12/10 22:08:24 Dropped index buck1_idx
2019/12/10 22:09:33 Flushed the bucket default
2019/12/10 22:09:36 Modified parameters of bucket default
2019/12/10 22:09:36 http://127.0.0.1:9000/pools/default/buckets/multibucket_test3
2019/12/10 22:09:36 &{DELETE http://127.0.0.1:9000/pools/default/buckets/multibucket_test3 HTTP/1.1 1 1 map[Authorization:[Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=] Content-Type:[application/x-www-form-urlencoded; charset=UTF-8]]   0 [] false 127.0.0.1:9000 map[] map[]  map[]      }
2019/12/10 22:09:36 &{404 Object Not Found 404 HTTP/1.1 1 1 map[X-Permitted-Cross-Domain-Policies:[none] X-Content-Type-Options:[nosniff] Server:[Couchbase Server] Date:[Tue, 10 Dec 2019 16:39:36 GMT] Content-Type:[text/plain] X-Xss-Protection:[1; mode=block] Pragma:[no-cache] Expires:[Thu, 01 Jan 1970 00:00:00 GMT] Content-Length:[31] Cache-Control:[no-cache,no-store,must-revalidate] X-Frame-Options:[DENY]] 0xc03a8a2dc0 31 [] false false map[] 0xc02b528a00 }
2019/12/10 22:09:36 DeleteBucket failed for bucket multibucket_test3 
2019/12/10 22:09:36 Deleted bucket multibucket_test3
2019/12/10 22:09:36 Created bucket multibucket_test3
2019/12/10 22:10:06 Setting JSON docs in KV
2019/12/10 22:11:30 Created the secondary index buck2_idx. Waiting for it become active
2019/12/10 22:11:30 Index is now active
2019/12/10 22:11:46 Created the secondary index buck1_idx
2019/12/10 22:11:48 Dropping the secondary index buck2_idx
2019/12/10 22:11:48 Index dropped
2019/12/10 22:11:48 Index is now active
2019/12/10 22:11:49 Using n1ql client
2019/12/10 22:11:51 Expected and Actual scan responses are the same
2019/12/10 22:11:51 Number of docScanResults and scanResults = 100000 and 100000
2019/12/10 22:11:53 Deleted bucket multibucket_test3
2019/12/10 22:13:02 Flushed the bucket default
--- PASS: TestDropBucket2Index_Bucket1IndexBuilding (277.58s)
=== RUN   TestDeleteBucketWhileInitialIndexBuild
2019/12/10 22:13:02 In TestDeleteBucketWhileInitialIndexBuild()
2019/12/10 22:13:02 ============== DBG: Drop all indexes in all buckets
2019/12/10 22:13:02 In DropAllSecondaryIndexes()
2019/12/10 22:13:02 Index found:  buck1_idx
2019/12/10 22:13:02 Dropped index buck1_idx
2019/12/10 22:13:02 ============== DBG: Flush bucket default
2019/12/10 22:14:10 Flushed the bucket default
2019/12/10 22:14:13 Modified parameters of bucket default
2019/12/10 22:14:13 ============== DBG: Delete bucket testbucket2
2019/12/10 22:14:13 http://127.0.0.1:9000/pools/default/buckets/testbucket2
2019/12/10 22:14:13 &{DELETE http://127.0.0.1:9000/pools/default/buckets/testbucket2 HTTP/1.1 1 1 map[Authorization:[Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=] Content-Type:[application/x-www-form-urlencoded; charset=UTF-8]]   0 [] false 127.0.0.1:9000 map[] map[]  map[]      }
2019/12/10 22:14:13 &{404 Object Not Found 404 HTTP/1.1 1 1 map[X-Xss-Protection:[1; mode=block] X-Permitted-Cross-Domain-Policies:[none] X-Content-Type-Options:[nosniff] Server:[Couchbase Server] Expires:[Thu, 01 Jan 1970 00:00:00 GMT] Content-Type:[text/plain] Content-Length:[31] X-Frame-Options:[DENY] Pragma:[no-cache] Date:[Tue, 10 Dec 2019 16:44:13 GMT] Cache-Control:[no-cache,no-store,must-revalidate]] 0xc07f39ec40 31 [] false false map[] 0xc00055e200 }
2019/12/10 22:14:13 DeleteBucket failed for bucket testbucket2 
2019/12/10 22:14:13 Deleted bucket testbucket2
2019/12/10 22:14:13 ============== DBG: Create bucket testbucket2
2019/12/10 22:14:14 Created bucket testbucket2
2019/12/10 22:14:17 Flush Enabled on bucket testbucket2
2019/12/10 22:15:24 Flushed the bucket testbucket2
2019/12/10 22:15:24 ============== DBG: Delete bucket testbucket3
2019/12/10 22:15:24 http://127.0.0.1:9000/pools/default/buckets/testbucket3
2019/12/10 22:15:24 &{DELETE http://127.0.0.1:9000/pools/default/buckets/testbucket3 HTTP/1.1 1 1 map[Authorization:[Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=] Content-Type:[application/x-www-form-urlencoded; charset=UTF-8]]   0 [] false 127.0.0.1:9000 map[] map[]  map[]      }
2019/12/10 22:15:24 &{404 Object Not Found 404 HTTP/1.1 1 1 map[X-Permitted-Cross-Domain-Policies:[none] X-Content-Type-Options:[nosniff] Pragma:[no-cache] Expires:[Thu, 01 Jan 1970 00:00:00 GMT] Date:[Tue, 10 Dec 2019 16:45:24 GMT] X-Xss-Protection:[1; mode=block] Server:[Couchbase Server] Content-Type:[text/plain] Content-Length:[31] Cache-Control:[no-cache,no-store,must-revalidate] X-Frame-Options:[DENY]] 0xc021f8d800 31 [] false false map[] 0xc00b59c300 }
2019/12/10 22:15:24 DeleteBucket failed for bucket testbucket3 
2019/12/10 22:15:24 Deleted bucket testbucket3
2019/12/10 22:15:24 ============== DBG: Create bucket testbucket3
2019/12/10 22:15:24 Created bucket testbucket3
2019/12/10 22:15:27 Flush Enabled on bucket testbucket3
2019/12/10 22:16:35 Flushed the bucket testbucket3
2019/12/10 22:16:35 ============== DBG: Delete bucket testbucket4
2019/12/10 22:16:35 http://127.0.0.1:9000/pools/default/buckets/testbucket4
2019/12/10 22:16:35 &{DELETE http://127.0.0.1:9000/pools/default/buckets/testbucket4 HTTP/1.1 1 1 map[Authorization:[Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=] Content-Type:[application/x-www-form-urlencoded; charset=UTF-8]]   0 [] false 127.0.0.1:9000 map[] map[]  map[]      }
2019/12/10 22:16:35 &{404 Object Not Found 404 HTTP/1.1 1 1 map[X-Xss-Protection:[1; mode=block] X-Content-Type-Options:[nosniff] Server:[Couchbase Server] Expires:[Thu, 01 Jan 1970 00:00:00 GMT] Date:[Tue, 10 Dec 2019 16:46:35 GMT] Content-Length:[31] X-Permitted-Cross-Domain-Policies:[none] X-Frame-Options:[DENY] Pragma:[no-cache] Content-Type:[text/plain] Cache-Control:[no-cache,no-store,must-revalidate]] 0xc01425b0c0 31 [] false false map[] 0xc00b59c000 }
2019/12/10 22:16:35 DeleteBucket failed for bucket testbucket4 
2019/12/10 22:16:35 Deleted bucket testbucket4
2019/12/10 22:16:35 ============== DBG: Create bucket testbucket4
2019/12/10 22:16:35 Created bucket testbucket4
2019/12/10 22:16:38 Flush Enabled on bucket testbucket4
2019/12/10 22:17:47 Flushed the bucket testbucket4
2019/12/10 22:18:17 Generating docs and Populating all the buckets
2019/12/10 22:18:18 ============== DBG: Creating docs in bucket default
2019/12/10 22:18:18 ============== DBG: Creating index bucket1_age in bucket default
2019/12/10 22:18:27 Created the secondary index bucket1_age. Waiting for it become active
2019/12/10 22:18:27 Index is now active
2019/12/10 22:18:27 ============== DBG: Creating index bucket1_gender in bucket default
2019/12/10 22:18:35 Created the secondary index bucket1_gender. Waiting for it become active
2019/12/10 22:18:35 Index is now active
2019/12/10 22:18:35 ============== DBG: Creating docs in bucket testbucket2
2019/12/10 22:18:37 ============== DBG: Creating index bucket2_city in bucket testbucket2
2019/12/10 22:18:45 Created the secondary index bucket2_city. Waiting for it become active
2019/12/10 22:18:45 Index is now active
2019/12/10 22:18:45 ============== DBG: Creating index bucket2_company in bucket testbucket2
2019/12/10 22:18:54 Created the secondary index bucket2_company. Waiting for it become active
2019/12/10 22:18:54 Index is now active
2019/12/10 22:18:54 ============== DBG: Creating docs in bucket testbucket3
2019/12/10 22:18:55 ============== DBG: Creating index bucket3_gender in bucket testbucket3
2019/12/10 22:19:04 Created the secondary index bucket3_gender. Waiting for it become active
2019/12/10 22:19:04 Index is now active
2019/12/10 22:19:04 ============== DBG: Creating index bucket3_address in bucket testbucket3
2019/12/10 22:19:14 Created the secondary index bucket3_address. Waiting for it become active
2019/12/10 22:19:14 Index is now active
2019/12/10 22:19:14 ============== DBG: First bucket scan:: Scanning index bucket1_age in bucket default
2019/12/10 22:19:14 Using n1ql client
2019/12/10 22:19:14 ============== DBG: First bucket scan:: Expected results = 310 Actual results = 310
2019/12/10 22:19:14 Expected and Actual scan responses are the same
2019/12/10 22:19:28 ============== DBG: Creating 50K docs in bucket testbucket4
2019/12/10 22:20:06 ============== DBG: Creating index bucket4_balance asynchronously in bucket testbucket4
2019/12/10 22:20:21 Created the secondary index bucket4_balance
2019/12/10 22:20:23 ============== DBG: Deleting bucket testbucket4
2019/12/10 22:20:26 Deleted bucket testbucket4
2019/12/10 22:20:26 ============== DBG: First bucket scan:: Scanning index bucket1_age in bucket default
2019/12/10 22:20:26 Using n1ql client
2019/12/10 22:20:26 ============== DBG: First bucket scan:: Expected results = 310 Actual results = 310
2019/12/10 22:20:26 Expected and Actual scan responses are the same
2019/12/10 22:20:26 ============== DBG: Second bucket scan:: Scanning index bucket2_city in bucket testbucket2
2019/12/10 22:20:26 Using n1ql client
2019-12-10T22:20:26.764+05:30 [Info] GsiClient::UpdateUsecjson: using collatejson as data format between indexer and GsiClient
2019-12-10T22:20:26.765+05:30 [Info] GSIC[default/testbucket2-1575996626757811387] started ...
2019/12/10 22:20:26 ============== DBG: Second bucket scan:: Expected results = 423 Actual results = 423
2019/12/10 22:20:26 Expected and Actual scan responses are the same
2019/12/10 22:20:26 ============== DBG: Third bucket scan:: Scanning index bucket3_gender in bucket testbucket3
2019/12/10 22:20:26 Using n1ql client
2019-12-10T22:20:26.797+05:30 [Info] GsiClient::UpdateUsecjson: using collatejson as data format between indexer and GsiClient
2019-12-10T22:20:26.797+05:30 [Info] GSIC[default/testbucket3-1575996626788435735] started ...
2019/12/10 22:20:26 ============== DBG: Third bucket scan:: Expected results = 521 Actual results = 521
2019/12/10 22:20:26 Expected and Actual scan responses are the same
2019/12/10 22:20:26 ============== DBG: Deleting buckets testbucket2 testbucket3 testbucket4
2019/12/10 22:20:29 Deleted bucket testbucket2
2019/12/10 22:20:31 Deleted bucket testbucket3
2019/12/10 22:20:31 http://127.0.0.1:9000/pools/default/buckets/testbucket4
2019/12/10 22:20:31 &{DELETE http://127.0.0.1:9000/pools/default/buckets/testbucket4 HTTP/1.1 1 1 map[Authorization:[Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=] Content-Type:[application/x-www-form-urlencoded; charset=UTF-8]]   0 [] false 127.0.0.1:9000 map[] map[]  map[]      }
2019/12/10 22:20:31 &{404 Object Not Found 404 HTTP/1.1 1 1 map[Pragma:[no-cache] Date:[Tue, 10 Dec 2019 16:50:31 GMT] Content-Type:[text/plain] Content-Length:[31] Cache-Control:[no-cache,no-store,must-revalidate] X-Xss-Protection:[1; mode=block] X-Permitted-Cross-Domain-Policies:[none] X-Content-Type-Options:[nosniff] X-Frame-Options:[DENY] Server:[Couchbase Server] Expires:[Thu, 01 Jan 1970 00:00:00 GMT]] 0xc0301c8cc0 31 [] false false map[] 0xc029979400 }
2019/12/10 22:20:31 DeleteBucket failed for bucket testbucket4 
2019/12/10 22:20:31 Deleted bucket testbucket4
2019/12/10 22:20:34 Modified parameters of bucket default
--- PASS: TestDeleteBucketWhileInitialIndexBuild (482.56s)
=== RUN   TestWherClause_UpdateDocument
2019/12/10 22:21:04 In TestWherClause_UpdateDocument()
2019/12/10 22:21:04 In DropAllSecondaryIndexes()
2019/12/10 22:21:04 Index found:  bucket1_age
2019/12/10 22:21:04 Dropped index bucket1_age
2019/12/10 22:21:04 Index found:  bucket1_gender
2019/12/10 22:21:05 Dropped index bucket1_gender
2019/12/10 22:22:14 Flushed the bucket default
2019/12/10 22:22:16 Setting JSON docs in KV
2019/12/10 22:22:29 Created the secondary index id_ageGreaterThan40. Waiting for it become active
2019/12/10 22:22:29 Index is now active
2019/12/10 22:22:29 Using n1ql client
2019/12/10 22:22:29 Expected and Actual scan responses are the same
2019/12/10 22:22:29 Number of docScanResults and scanResults = 6008 and 6008
2019/12/10 22:22:32 Using n1ql client
2019/12/10 22:22:33 Expected and Actual scan responses are the same
2019/12/10 22:22:33 Number of docScanResults and scanResults = 2008 and 2008
--- PASS: TestWherClause_UpdateDocument (88.45s)
=== RUN   TestDeferFalse
2019/12/10 22:22:33 In TestDeferFalse()
2019/12/10 22:22:36 Setting JSON docs in KV
2019/12/10 22:22:52 Created the secondary index index_deferfalse1. Waiting for it become active
2019/12/10 22:22:52 Index is now active
2019/12/10 22:22:52 Using n1ql client
2019/12/10 22:22:53 Expected and Actual scan responses are the same
--- PASS: TestDeferFalse (19.94s)
=== RUN   TestDeferFalse_CloseClientConnection
2019/12/10 22:22:53 In TestDeferFalse_CloseClientConnection()
2019/12/10 22:22:53 In CloseClientThread
2019/12/10 22:22:53 In CreateIndexThread
2019/12/10 22:22:55 Create Index call failed as expected due to error : Terminate Request due to client termination
2019-12-10T22:22:55.208+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:9106.  Error = read tcp 127.0.0.1:60396->127.0.0.1:9106: use of closed network connection. Kill Pipe.
2019/12/10 22:22:55 Waiting for index to go active ...
2019/12/10 22:22:56 Waiting for index to go active ...
2019/12/10 22:22:57 Waiting for index to go active ...
2019/12/10 22:22:58 Waiting for index to go active ...
2019/12/10 22:22:59 Waiting for index to go active ...
2019/12/10 22:23:00 Waiting for index to go active ...
2019/12/10 22:23:01 Index is now active
2019/12/10 22:23:01 Using n1ql client
2019/12/10 22:23:01 Expected and Actual scan responses are the same
--- PASS: TestDeferFalse_CloseClientConnection (8.37s)
=== RUN   TestOrphanIndexCleanup
2019-12-10T22:23:01.487+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:9106.  Error = read tcp 127.0.0.1:60430->127.0.0.1:9106: use of closed network connection. Kill Pipe.
2019/12/10 22:23:01 In DropAllSecondaryIndexes()
2019/12/10 22:23:01 Index found:  index_deferfalse1
2019/12/10 22:23:01 Dropped index index_deferfalse1
2019/12/10 22:23:01 Index found:  index_deferfalse2
2019/12/10 22:23:01 Dropped index index_deferfalse2
2019/12/10 22:23:01 Index found:  id_ageGreaterThan40
2019/12/10 22:23:02 Dropped index id_ageGreaterThan40
2019/12/10 22:23:19 Created the secondary index idx1_age_regular. Waiting for it become active
2019/12/10 22:23:19 Index is now active
2019/12/10 22:23:26 Created the secondary index idx2_company_regular. Waiting for it become active
2019/12/10 22:23:26 Index is now active
2019/12/10 22:23:26 Using n1ql client
2019/12/10 22:23:26 Query on idx1_age_regular is successful
2019/12/10 22:23:26 Using n1ql client
2019/12/10 22:23:26 Query on idx2_company_regular is successful
Restarting indexer process ...
2019/12/10 22:23:26 []
2019-12-10T22:23:26.977+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:9106.  Error = EOF. Kill Pipe.
2019-12-10T22:23:26.977+05:30 [Error] WatcherServer.runOnce() : Watcher terminated unexpectedly.
2019-12-10T22:23:26.977+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:9106.  Error = EOF. Kill Pipe.
2019-12-10T22:23:26.978+05:30 [Error] WatcherServer.runOnce() : Watcher terminated unexpectedly.
2019/12/10 22:23:56 Using n1ql client
2019-12-10T22:23:56.832+05:30 [Error] transport error between 127.0.0.1:42280->127.0.0.1:9107: write tcp 127.0.0.1:42280->127.0.0.1:9107: write: broken pipe
2019-12-10T22:23:56.832+05:30 [Error] [GsiScanClient:"127.0.0.1:9107"] -3519224342293457717 request transport failed `write tcp 127.0.0.1:42280->127.0.0.1:9107: write: broken pipe`
2019-12-10T22:23:56.832+05:30 [Error] PickRandom: Fail to find indexer for all index partitions. Num partition 1.  Partition with instances 0 
2019/12/10 22:23:56 Query on idx1_age_regular is successful - after indexer restart.
2019/12/10 22:23:56 Using n1ql client
2019/12/10 22:23:56 Query on idx2_company_regular is successful - after indexer restart.
--- PASS: TestOrphanIndexCleanup (55.46s)
=== RUN   TestOrphanPartitionCleanup
Not running TestOrphanPartitionCleanup for forestdb
--- PASS: TestOrphanPartitionCleanup (0.00s)
=== RUN   TestIndexerSettings
2019/12/10 22:23:56 In TestIndexerSettings()
2019/12/10 22:23:56 DBG: ChangeIndexerSettings: nodes = [0xc015f809c0]
2019/12/10 22:23:56 DBG: ChangeIndexerSettings: adminurl = 127.0.0.1:9106 host 127.0.0.1 sport 9106
2019/12/10 22:23:56 Changing config key indexer.settings.max_cpu_percent to value 300
2019/12/10 22:23:56 DBG: ChangeIndexerSettings: nodes = [0xc050c58f40]
2019/12/10 22:23:56 DBG: ChangeIndexerSettings: adminurl = 127.0.0.1:9106 host 127.0.0.1 sport 9106
2019/12/10 22:23:56 Changing config key indexer.settings.inmemory_snapshot.interval to value 300
2019-12-10T22:23:57.036+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9106 connected=true
2019/12/10 22:23:57 DBG: ChangeIndexerSettings: nodes = [0xc015f80ac0]
2019/12/10 22:23:57 DBG: ChangeIndexerSettings: adminurl = 127.0.0.1:9106 host 127.0.0.1 sport 9106
2019/12/10 22:23:57 Changing config key indexer.settings.persisted_snapshot.interval to value 20000
2019-12-10T22:23:57.052+05:30 [Info] New settings received: 
{"indexer.api.enableTestServer":true,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfFname":"","indexer.settings.cpuProfile":false,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":300,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":300,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfFname":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":250,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.recovery.max_rollbacks":2,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":0,"indexer.settings.send_buffer_size":1024,"indexer.settings.sliceBufSize":50000,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"forestdb","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.wal_size":4096,"projector.settings.log_level":"info","queryport.client.settings.backfillLimit":0,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100}
2019-12-10T22:23:57.079+05:30 [Info] New settings received: 
{"indexer.api.enableTestServer":true,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfFname":"","indexer.settings.cpuProfile":false,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":300,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":300,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfFname":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":250,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":20000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.recovery.max_rollbacks":2,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":0,"indexer.settings.send_buffer_size":1024,"indexer.settings.sliceBufSize":50000,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"forestdb","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.wal_size":4096,"projector.settings.log_level":"info","queryport.client.settings.backfillLimit":0,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100}
2019-12-10T22:23:57.110+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9106 connected=true
2019/12/10 22:23:57 DBG: ChangeIndexerSettings: nodes = [0xc015f80bc0]
2019/12/10 22:23:57 DBG: ChangeIndexerSettings: adminurl = 127.0.0.1:9106 host 127.0.0.1 sport 9106
2019/12/10 22:23:57 Changing config key indexer.settings.recovery.max_rollbacks to value 3
2019-12-10T22:23:57.160+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9106 connected=true
2019/12/10 22:23:57 DBG: ChangeIndexerSettings: nodes = [0xc050c59040]
2019/12/10 22:23:57 DBG: ChangeIndexerSettings: adminurl = 127.0.0.1:9106 host 127.0.0.1 sport 9106
2019/12/10 22:23:57 Changing config key indexer.settings.log_level to value error
2019-12-10T22:23:57.160+05:30 [Info] New settings received: 
{"indexer.api.enableTestServer":true,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfFname":"","indexer.settings.cpuProfile":false,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":300,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":300,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfFname":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":250,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":20000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.recovery.max_rollbacks":3,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":0,"indexer.settings.send_buffer_size":1024,"indexer.settings.sliceBufSize":50000,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"forestdb","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.wal_size":4096,"projector.settings.log_level":"info","queryport.client.settings.backfillLimit":0,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100}
--- PASS: TestIndexerSettings (0.28s)
=== RUN   TestRestoreDefaultSettings
2019/12/10 22:23:57 In TestIndexerSettings_RestoreDefault()
2019-12-10T22:23:57.223+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9106 connected=true
2019/12/10 22:23:57 DBG: ChangeIndexerSettings: nodes = [0xc015f80cc0]
2019/12/10 22:23:57 DBG: ChangeIndexerSettings: adminurl = 127.0.0.1:9106 host 127.0.0.1 sport 9106
2019/12/10 22:23:57 Changing config key indexer.settings.max_cpu_percent to value 0
2019-12-10T22:23:57.234+05:30 [Info] New settings received: 
{"indexer.api.enableTestServer":true,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfFname":"","indexer.settings.cpuProfile":false,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":300,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"error","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":300,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfFname":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":250,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":20000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.recovery.max_rollbacks":3,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":0,"indexer.settings.send_buffer_size":1024,"indexer.settings.sliceBufSize":50000,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"forestdb","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.wal_size":4096,"projector.settings.log_level":"info","queryport.client.settings.backfillLimit":0,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100}
2019-12-10T22:23:57.310+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9106 connected=true
2019/12/10 22:23:57 DBG: ChangeIndexerSettings: nodes = [0xc050c59140]
2019/12/10 22:23:57 DBG: ChangeIndexerSettings: adminurl = 127.0.0.1:9106 host 127.0.0.1 sport 9106
2019/12/10 22:23:57 Changing config key indexer.settings.inmemory_snapshot.interval to value 200
2019-12-10T22:23:57.323+05:30 [Info] New settings received: 
{"indexer.api.enableTestServer":true,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfFname":"","indexer.settings.cpuProfile":false,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":300,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"error","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfFname":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":250,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":20000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.recovery.max_rollbacks":3,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":0,"indexer.settings.send_buffer_size":1024,"indexer.settings.sliceBufSize":50000,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"forestdb","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.wal_size":4096,"projector.settings.log_level":"info","queryport.client.settings.backfillLimit":0,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100}
2019-12-10T22:23:57.368+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9106 connected=true
2019/12/10 22:23:57 DBG: ChangeIndexerSettings: nodes = [0xc037e24a80]
2019/12/10 22:23:57 DBG: ChangeIndexerSettings: adminurl = 127.0.0.1:9106 host 127.0.0.1 sport 9106
2019/12/10 22:23:57 Changing config key indexer.settings.persisted_snapshot.interval to value 5000
2019-12-10T22:23:57.373+05:30 [Info] New settings received: 
{"indexer.api.enableTestServer":true,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfFname":"","indexer.settings.cpuProfile":false,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"error","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfFname":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":250,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":20000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.recovery.max_rollbacks":3,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":0,"indexer.settings.send_buffer_size":1024,"indexer.settings.sliceBufSize":50000,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"forestdb","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.wal_size":4096,"projector.settings.log_level":"info","queryport.client.settings.backfillLimit":0,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100}
2019-12-10T22:23:57.433+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9106 connected=true
2019/12/10 22:23:57 DBG: ChangeIndexerSettings: nodes = [0xc015f80e40]
2019/12/10 22:23:57 DBG: ChangeIndexerSettings: adminurl = 127.0.0.1:9106 host 127.0.0.1 sport 9106
2019/12/10 22:23:57 Changing config key indexer.settings.recovery.max_rollbacks to value 5
2019-12-10T22:23:57.443+05:30 [Info] New settings received: 
{"indexer.api.enableTestServer":true,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfFname":"","indexer.settings.cpuProfile":false,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"error","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfFname":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":250,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.recovery.max_rollbacks":3,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":0,"indexer.settings.send_buffer_size":1024,"indexer.settings.sliceBufSize":50000,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"forestdb","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.wal_size":4096,"projector.settings.log_level":"info","queryport.client.settings.backfillLimit":0,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100}
2019-12-10T22:23:57.497+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9106 connected=true
2019/12/10 22:23:57 DBG: ChangeIndexerSettings: nodes = [0xc015f80f40]
2019/12/10 22:23:57 DBG: ChangeIndexerSettings: adminurl = 127.0.0.1:9106 host 127.0.0.1 sport 9106
2019/12/10 22:23:57 Changing config key indexer.settings.log_level to value info
2019-12-10T22:23:57.514+05:30 [Info] New settings received: 
{"indexer.api.enableTestServer":true,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfFname":"","indexer.settings.cpuProfile":false,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"error","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfFname":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":250,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.recovery.max_rollbacks":5,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":0,"indexer.settings.send_buffer_size":1024,"indexer.settings.sliceBufSize":50000,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"forestdb","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.wal_size":4096,"projector.settings.log_level":"info","queryport.client.settings.backfillLimit":0,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100}
2019-12-10T22:23:57.558+05:30 [Info] New settings received: 
{"indexer.api.enableTestServer":true,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfFname":"","indexer.settings.cpuProfile":false,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfFname":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":250,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.recovery.max_rollbacks":5,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":0,"indexer.settings.send_buffer_size":1024,"indexer.settings.sliceBufSize":50000,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"forestdb","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.wal_size":4096,"projector.settings.log_level":"info","queryport.client.settings.backfillLimit":0,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100}
--- PASS: TestRestoreDefaultSettings (0.35s)
=== RUN   TestStat_ItemsCount
2019/12/10 22:23:57 In TestStat_ItemsCount()
2019/12/10 22:23:57 In DropAllSecondaryIndexes()
2019/12/10 22:23:57 Index found:  idx1_age_regular
2019-12-10T22:23:57.862+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:23:57.862+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:23:57.867+05:30 [Info] switched currmeta from 1260 -> 1261 force true 
2019-12-10T22:23:57.870+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:23:57.870+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:23:57.872+05:30 [Info] switched currmeta from 1256 -> 1257 force true 
2019-12-10T22:23:57.964+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:23:57.964+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:23:57.967+05:30 [Info] switched currmeta from 1257 -> 1257 force true 
2019-12-10T22:23:57.968+05:30 [Info] metadata provider version changed 1261 -> 1262
2019-12-10T22:23:57.968+05:30 [Info] switched currmeta from 1261 -> 1262 force false 
2019-12-10T22:23:57.968+05:30 [Info] DropIndex 11308361974312744876 - elapsed(392.794976ms), err()
2019/12/10 22:23:57 Dropped index idx1_age_regular
2019/12/10 22:23:57 Index found:  idx2_company_regular
2019-12-10T22:23:57.970+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:23:57.970+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:23:57.975+05:30 [Info] switched currmeta from 1262 -> 1262 force true 
2019-12-10T22:23:58.083+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:23:58.083+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:23:58.083+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:23:58.083+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:23:58.089+05:30 [Info] switched currmeta from 1262 -> 1263 force true 
2019-12-10T22:23:58.090+05:30 [Info] switched currmeta from 1257 -> 1259 force true 
2019-12-10T22:23:58.135+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:23:58.135+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:23:58.139+05:30 [Info] switched currmeta from 1259 -> 1259 force true 
2019-12-10T22:23:58.161+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:23:58.162+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:23:58.169+05:30 [Info] switched currmeta from 1263 -> 1263 force true 
2019-12-10T22:23:58.180+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:23:58.180+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:23:58.189+05:30 [Info] switched currmeta from 1259 -> 1259 force true 
2019-12-10T22:23:58.232+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:23:58.232+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:23:58.264+05:30 [Info] switched currmeta from 1263 -> 1263 force true 
2019-12-10T22:23:58.264+05:30 [Info] metadata provider version changed 1263 -> 1264
2019-12-10T22:23:58.265+05:30 [Info] switched currmeta from 1263 -> 1264 force false 
2019-12-10T22:23:58.265+05:30 [Info] DropIndex 6910012219686121564 - elapsed(296.082422ms), err()
2019/12/10 22:23:58 Dropped index idx2_company_regular
2019/12/10 22:23:58 Emptying the default bucket
2019-12-10T22:23:58.331+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] active conns 0, free conns 1
2019-12-10T22:23:58.385+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:23:58.385+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:23:58.401+05:30 [Info] switched currmeta from 1259 -> 1260 force true 
2019-12-10T22:23:58.411+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:23:58.412+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:23:58.414+05:30 [Info] switched currmeta from 1264 -> 1264 force true 
2019-12-10T22:24:00.524+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:24:00.550+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:24:00.550+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:24:00.551+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:24:00.551+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:24:00.552+05:30 [Info] switched currmeta from 1260 -> 1260 force true 
2019-12-10T22:24:00.553+05:30 [Info] switched currmeta from 1264 -> 1264 force true 
2019/12/10 22:24:01 Flush Enabled on bucket default
2019-12-10T22:24:02.561+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:24:02.629+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:24:02.630+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:24:02.665+05:30 [Info] switched currmeta from 1264 -> 1264 force true 
2019-12-10T22:24:02.669+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:24:02.669+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:24:02.672+05:30 [Info] switched currmeta from 1260 -> 1260 force true 
2019-12-10T22:24:05.398+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:24:05.434+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:24:05.434+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:24:05.440+05:30 [Info] switched currmeta from 1264 -> 1264 force true 
2019-12-10T22:24:05.442+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:24:05.442+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:24:05.449+05:30 [Info] switched currmeta from 1260 -> 1260 force true 
2019-12-10T22:24:05.527+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:24:05.558+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:24:05.558+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:24:05.559+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:24:05.559+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:24:05.560+05:30 [Info] switched currmeta from 1264 -> 1264 force true 
2019-12-10T22:24:05.563+05:30 [Info] switched currmeta from 1260 -> 1260 force true 
2019-12-10T22:24:10.398+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:24:10.439+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:24:10.439+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:24:10.444+05:30 [Info] switched currmeta from 1260 -> 1260 force true 
2019-12-10T22:24:10.447+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:24:10.447+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:24:10.449+05:30 [Info] switched currmeta from 1264 -> 1264 force true 
2019-12-10T22:24:11.340+05:30 [Info] GSIC[default/default-1575992531337069731] logstats "default" {"gsi_scan_count":73,"gsi_scan_duration":58566424629,"gsi_throttle_duration":576322960,"gsi_prime_duration":1781640088,"gsi_blocked_duration":465664674,"gsi_total_temp_files":0}
2019-12-10T22:24:12.553+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:24:12.612+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:24:12.612+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:24:12.615+05:30 [Info] switched currmeta from 1264 -> 1264 force true 
2019-12-10T22:24:12.616+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:24:12.616+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:24:12.618+05:30 [Info] switched currmeta from 1260 -> 1260 force true 
2019-12-10T22:24:38.006+05:30 [Info] connected with 1 indexers
2019-12-10T22:24:38.006+05:30 [Info] client load stats {}
2019-12-10T22:24:38.006+05:30 [Info] client hit stats {}
2019-12-10T22:24:38.006+05:30 [Info] client pending item stats {}
2019-12-10T22:24:38.006+05:30 [Info] client stats current {}
2019-12-10T22:24:38.009+05:30 [Info] num concurrent scans {0}
2019-12-10T22:24:38.009+05:30 [Info] average scan response {65 ms}
2019-12-10T22:24:40.536+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] active conns 0, free conns 1
2019-12-10T22:24:55.866+05:30 [Info] connected with 1 indexers
2019-12-10T22:24:55.866+05:30 [Info] client load stats {}
2019-12-10T22:24:55.866+05:30 [Info] client hit stats {}
2019-12-10T22:24:55.866+05:30 [Info] client pending item stats {}
2019-12-10T22:24:55.866+05:30 [Info] client stats current {}
2019-12-10T22:24:55.873+05:30 [Info] num concurrent scans {0}
2019-12-10T22:24:55.874+05:30 [Info] average scan response {8 ms}
2019-12-10T22:24:58.349+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] active conns 0, free conns 1
2019/12/10 22:25:10 Flushed the bucket default
2019/12/10 22:25:15 Generating JSON docs
2019/12/10 22:25:15 Setting initial JSON docs in KV
2019/12/10 22:25:20 Creating a 2i
2019-12-10T22:25:20.935+05:30 [Info] send prepare create request to watcher 127.0.0.1:9106
2019-12-10T22:25:20.936+05:30 [Info] Indexer 127.0.0.1:9106 accept prepare request. Index (default, index_test1)
2019-12-10T22:25:21.024+05:30 [Info] Planner::processCreateToken: There is no create token to process for node 127.0.0.1:9001
2019-12-10T22:25:21.493+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node 127.0.0.1:9001
2019-12-10T22:25:21.584+05:30 [Info] ************ Indexer Layout *************
2019-12-10T22:25:21.584+05:30 [Info] Score: 0
2019-12-10T22:25:21.584+05:30 [Info] variation: 0
2019-12-10T22:25:21.584+05:30 [Info] ElapsedTime: 7ms
2019-12-10T22:25:21.584+05:30 [Info] ConvergenceTime: 2us
2019-12-10T22:25:21.584+05:30 [Info] Iteration: 0
2019-12-10T22:25:21.584+05:30 [Info] Move: 0
2019-12-10T22:25:21.584+05:30 [Info] ----------------------------------------
2019-12-10T22:25:21.584+05:30 [Info] Indexer Memory Mean 1258291200 (1.17188G)
2019-12-10T22:25:21.584+05:30 [Info] Indexer Memory Deviation 0 (0) (0.00%)
2019-12-10T22:25:21.584+05:30 [Info] Indexer Memory Utilization 0.8000
2019-12-10T22:25:21.584+05:30 [Info] Indexer CPU Mean 0.0000
2019-12-10T22:25:21.584+05:30 [Info] Indexer CPU Deviation 0.00 (0.00%)
2019-12-10T22:25:21.584+05:30 [Info] Indexer CPU Utilization 0.0000
2019-12-10T22:25:21.584+05:30 [Info] Indexer IO Mean 0.0000
2019-12-10T22:25:21.584+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2019-12-10T22:25:21.584+05:30 [Info] Indexer Drain Rate Mean 0.0000
2019-12-10T22:25:21.584+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2019-12-10T22:25:21.584+05:30 [Info] Indexer Scan Rate Mean 0.0000
2019-12-10T22:25:21.584+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2019-12-10T22:25:21.584+05:30 [Info] Indexer Data Size Mean 1258291200 (1.17188G)
2019-12-10T22:25:21.584+05:30 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2019-12-10T22:25:21.584+05:30 [Info] Total Index Data (from non-deleted node) 0
2019-12-10T22:25:21.585+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2019-12-10T22:25:21.585+05:30 [Info] No. Index (from non-deleted node) 0
2019-12-10T22:25:21.585+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2019-12-10T22:25:21.585+05:30 [Info] ----------------------------------------
2019-12-10T22:25:21.585+05:30 [Info] Number of indexes: 1
2019-12-10T22:25:21.585+05:30 [Info] Number of indexers: 1
2019-12-10T22:25:21.585+05:30 [Info] Avg Index Size: 1258291200 (1.17188G)
2019-12-10T22:25:21.585+05:30 [Info] Max Index Size: 1258291200 (1.17188G)
2019-12-10T22:25:21.585+05:30 [Info] Max Indexer Overhead: 0 (0)
2019-12-10T22:25:21.585+05:30 [Info] Avg Index Cpu: 0.0000
2019-12-10T22:25:21.585+05:30 [Info] Max Index Cpu: 0.0000
2019-12-10T22:25:21.585+05:30 [Info] Num Estimation Run: 1
2019-12-10T22:25:21.585+05:30 [Info] ----------------------------------------
2019-12-10T22:25:21.585+05:30 [Info] Memory Quota 1572864000 (1.46484G)
2019-12-10T22:25:21.585+05:30 [Info] CPU Quota 4
2019-12-10T22:25:21.585+05:30 [Info] Max Cpu Utilization -1
2019-12-10T22:25:21.585+05:30 [Info] Max Memory Utilization -1
2019-12-10T22:25:21.585+05:30 [Info] ----------------------------------------
2019-12-10T22:25:21.585+05:30 [Info] Total iteration: 0
2019-12-10T22:25:21.585+05:30 [Info] RandomSwap time: 0ns
2019-12-10T22:25:21.585+05:30 [Info] RandomSwap call: 0
2019-12-10T22:25:21.585+05:30 [Info] RandomSwap iteration: 0
2019-12-10T22:25:21.585+05:30 [Info] RandomMove time: 0ns
2019-12-10T22:25:21.585+05:30 [Info] RandomMove call: 0 (empty index 0)
2019-12-10T22:25:21.585+05:30 [Info] ExhaustMove time: 0ns
2019-12-10T22:25:21.585+05:30 [Info] ExhaustMove call: 0
2019-12-10T22:25:21.585+05:30 [Info] ExhaustSwap time: 0ns
2019-12-10T22:25:21.585+05:30 [Info] ExhaustSwap call: 0
2019-12-10T22:25:21.585+05:30 [Info] ----------------------------------------
2019-12-10T22:25:21.585+05:30 [Info] 
2019-12-10T22:25:21.585+05:30 [Info] Indexer serverGroup:Group 1, nodeId:127.0.0.1:9001, nodeUUID:280ac5e1026b443deea667c0a12beceb, useLiveData:true
2019-12-10T22:25:21.586+05:30 [Info] Indexer total memory:1258291200 (1.17188G), mem:1258291200 (1.17188G), overhead:0 (0), min:1258291200 (1.17188G), data:1258291200 (1.17188G) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:1
2019-12-10T22:25:21.586+05:30 [Info] Indexer isDeleted:false isNew:false exclude: meetConstraint:true
2019-12-10T22:25:21.586+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:25:21.586+05:30 [Info] 		Index name:index_test1 0, bucket:default, defnId:6948064276641493321, instId:1781406591464998797, Partition: 0, new/moved:true
2019-12-10T22:25:21.586+05:30 [Info] 		Index total memory:1258291200 (1.17188G), mem:1258291200 (1.17188G), overhead:0 (0), min:1258291200 (1.17188G), data:1258291200 (1.17188G) cpu:0.0000 io:0 (0) scan:0 drain:0
2019-12-10T22:25:21.586+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:25:21.586+05:30 [Info] ****************************************
2019-12-10T22:25:21.586+05:30 [Info] send commit create request to watcher 127.0.0.1:9106
2019-12-10T22:25:21.786+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:25:21.786+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:25:21.786+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:25:21.786+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:25:21.789+05:30 [Info] switched currmeta from 1260 -> 1262 force true 
2019-12-10T22:25:21.789+05:30 [Info] switched currmeta from 1264 -> 1266 force true 
2019-12-10T22:25:22.005+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:25:22.005+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:25:22.006+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:25:22.006+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:25:22.008+05:30 [Info] switched currmeta from 1262 -> 1263 force true 
2019-12-10T22:25:22.010+05:30 [Info] switched currmeta from 1266 -> 1268 force true 
2019-12-10T22:25:22.059+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:25:22.059+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:25:22.066+05:30 [Info] switched currmeta from 1268 -> 1268 force true 
2019-12-10T22:25:22.076+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:25:22.076+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:25:22.080+05:30 [Info] switched currmeta from 1263 -> 1264 force true 
2019-12-10T22:25:22.229+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:25:22.229+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:25:22.237+05:30 [Info] switched currmeta from 1268 -> 1269 force true 
2019-12-10T22:25:22.239+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:25:22.239+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:25:22.245+05:30 [Info] switched currmeta from 1264 -> 1265 force true 
2019-12-10T22:25:25.548+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:25:25.602+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:25:25.602+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:25:25.606+05:30 [Info] switched currmeta from 1265 -> 1265 force true 
2019-12-10T22:25:25.607+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:25:25.607+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:25:25.611+05:30 [Info] switched currmeta from 1269 -> 1269 force true 
2019-12-10T22:25:25.702+05:30 [Info] CreateIndex 6948064276641493321 default/index_test1 using:forestdb exprType:N1QL whereExpr:() secExprs:([`company`]) desc:[] isPrimary:false scheme:SINGLE  partitionKeys:([]) with: - elapsed(4.767352892s) err()
2019/12/10 22:25:25 Created the secondary index index_test1. Waiting for it become active
2019-12-10T22:25:25.702+05:30 [Info] metadata provider version changed 1269 -> 1270
2019-12-10T22:25:25.702+05:30 [Info] switched currmeta from 1269 -> 1270 force false 
2019/12/10 22:25:25 Index is now active
2019-12-10T22:25:25.753+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:25:25.753+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:25:25.761+05:30 [Info] switched currmeta from 1265 -> 1266 force true 
2019-12-10T22:25:25.767+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:25:25.767+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:25:25.768+05:30 [Info] switched currmeta from 1270 -> 1270 force true 
2019-12-10T22:25:27.630+05:30 [Info] Rollback time has changed for index inst 8231482558124385827. New rollback time 1575996807857833855
2019-12-10T22:25:27.631+05:30 [Info] Rollback time has changed for index inst 8231482558124385827. New rollback time 1575996807857833855
2019-12-10T22:25:30.546+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:25:30.574+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:25:30.574+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:25:30.576+05:30 [Info] switched currmeta from 1270 -> 1270 force true 
2019-12-10T22:25:30.576+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:25:30.579+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:25:30.582+05:30 [Info] switched currmeta from 1266 -> 1266 force true 
2019-12-10T22:25:35.546+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:25:35.579+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:25:35.579+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:25:35.581+05:30 [Info] switched currmeta from 1270 -> 1270 force true 
2019-12-10T22:25:35.581+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:25:35.581+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:25:35.583+05:30 [Info] switched currmeta from 1266 -> 1266 force true 
2019/12/10 22:25:35 items_count stat is 10000
--- PASS: TestStat_ItemsCount (98.15s)
=== RUN   TestRangeArrayIndex_Distinct
2019/12/10 22:25:35 In TestRangeArrayIndex_Distinct()
2019/12/10 22:25:35 In DropAllSecondaryIndexes()
2019/12/10 22:25:35 Index found:  index_test1
2019-12-10T22:25:35.878+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:25:35.878+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:25:35.878+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:25:35.878+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:25:35.882+05:30 [Info] switched currmeta from 1266 -> 1267 force true 
2019-12-10T22:25:35.882+05:30 [Info] switched currmeta from 1270 -> 1271 force true 
2019-12-10T22:25:35.929+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:25:35.929+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:25:35.931+05:30 [Info] switched currmeta from 1267 -> 1267 force true 
2019-12-10T22:25:35.934+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:25:35.934+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:25:35.935+05:30 [Info] switched currmeta from 1271 -> 1271 force true 
2019-12-10T22:25:35.938+05:30 [Info] metadata provider version changed 1271 -> 1272
2019-12-10T22:25:35.938+05:30 [Info] switched currmeta from 1271 -> 1272 force false 
2019-12-10T22:25:35.938+05:30 [Info] DropIndex 6948064276641493321 - elapsed(210.172473ms), err()
2019/12/10 22:25:35 Dropped index index_test1
2019-12-10T22:25:36.079+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:25:36.079+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:25:36.084+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:25:36.084+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:25:36.092+05:30 [Info] switched currmeta from 1272 -> 1272 force true 
2019-12-10T22:25:36.093+05:30 [Info] switched currmeta from 1267 -> 1268 force true 
2019-12-10T22:25:37.563+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:25:37.628+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:25:37.628+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:25:37.631+05:30 [Info] switched currmeta from 1268 -> 1268 force true 
2019-12-10T22:25:37.637+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:25:37.637+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:25:37.640+05:30 [Info] switched currmeta from 1272 -> 1272 force true 
2019-12-10T22:25:38.006+05:30 [Info] connected with 1 indexers
2019-12-10T22:25:38.006+05:30 [Info] client load stats {}
2019-12-10T22:25:38.006+05:30 [Info] client hit stats {}
2019-12-10T22:25:38.006+05:30 [Info] client pending item stats {}
2019-12-10T22:25:38.006+05:30 [Info] client stats current {}
2019-12-10T22:25:38.009+05:30 [Info] num concurrent scans {0}
2019-12-10T22:25:38.009+05:30 [Info] average scan response {65 ms}
2019-12-10T22:25:40.401+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:25:40.446+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:25:40.446+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:25:40.448+05:30 [Info] switched currmeta from 1272 -> 1272 force true 
2019-12-10T22:25:40.451+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:25:40.451+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:25:40.455+05:30 [Info] switched currmeta from 1268 -> 1268 force true 
2019-12-10T22:25:40.546+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:25:40.555+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] active conns 0, free conns 1
2019-12-10T22:25:40.575+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:25:40.575+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:25:40.579+05:30 [Info] switched currmeta from 1272 -> 1272 force true 
2019-12-10T22:25:40.582+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:25:40.582+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:25:40.584+05:30 [Info] switched currmeta from 1268 -> 1268 force true 
2019-12-10T22:25:45.405+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:25:45.492+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:25:45.492+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:25:45.496+05:30 [Info] switched currmeta from 1268 -> 1268 force true 
2019-12-10T22:25:45.505+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:25:45.505+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:25:45.509+05:30 [Info] switched currmeta from 1272 -> 1272 force true 
2019-12-10T22:25:47.554+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:25:47.600+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:25:47.600+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:25:47.611+05:30 [Info] switched currmeta from 1272 -> 1272 force true 
2019-12-10T22:25:47.628+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:25:47.628+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:25:47.635+05:30 [Info] switched currmeta from 1268 -> 1268 force true 
2019-12-10T22:25:55.866+05:30 [Info] connected with 1 indexers
2019-12-10T22:25:55.866+05:30 [Info] client load stats {}
2019-12-10T22:25:55.866+05:30 [Info] client hit stats {}
2019-12-10T22:25:55.866+05:30 [Info] client pending item stats {}
2019-12-10T22:25:55.866+05:30 [Info] client stats current {}
2019-12-10T22:25:55.874+05:30 [Info] num concurrent scans {0}
2019-12-10T22:25:55.874+05:30 [Info] average scan response {8 ms}
2019-12-10T22:25:58.367+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] active conns 0, free conns 1
2019-12-10T22:26:38.006+05:30 [Info] connected with 1 indexers
2019-12-10T22:26:38.006+05:30 [Info] client load stats {}
2019-12-10T22:26:38.006+05:30 [Info] client hit stats {}
2019-12-10T22:26:38.006+05:30 [Info] client pending item stats {}
2019-12-10T22:26:38.006+05:30 [Info] client stats current {}
2019-12-10T22:26:38.009+05:30 [Info] num concurrent scans {0}
2019-12-10T22:26:38.009+05:30 [Info] average scan response {65 ms}
2019-12-10T22:26:40.576+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] active conns 0, free conns 1
2019/12/10 22:26:44 Flushed the bucket default
2019-12-10T22:26:45.568+05:30 [Info] send prepare create request to watcher 127.0.0.1:9106
2019-12-10T22:26:45.569+05:30 [Info] Indexer 127.0.0.1:9106 accept prepare request. Index (default, arridx_friends)
2019-12-10T22:26:45.629+05:30 [Info] Planner::processCreateToken: There is no create token to process for node 127.0.0.1:9001
2019-12-10T22:26:46.060+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node 127.0.0.1:9001
2019-12-10T22:26:46.210+05:30 [Info] ************ Indexer Layout *************
2019-12-10T22:26:46.210+05:30 [Info] Score: 0
2019-12-10T22:26:46.210+05:30 [Info] variation: 0
2019-12-10T22:26:46.210+05:30 [Info] ElapsedTime: 66ms
2019-12-10T22:26:46.210+05:30 [Info] ConvergenceTime: 2us
2019-12-10T22:26:46.210+05:30 [Info] Iteration: 0
2019-12-10T22:26:46.210+05:30 [Info] Move: 0
2019-12-10T22:26:46.210+05:30 [Info] ----------------------------------------
2019-12-10T22:26:46.210+05:30 [Info] Indexer Memory Mean 1258291200 (1.17188G)
2019-12-10T22:26:46.210+05:30 [Info] Indexer Memory Deviation 0 (0) (0.00%)
2019-12-10T22:26:46.210+05:30 [Info] Indexer Memory Utilization 0.8000
2019-12-10T22:26:46.210+05:30 [Info] Indexer CPU Mean 0.0000
2019-12-10T22:26:46.210+05:30 [Info] Indexer CPU Deviation 0.00 (0.00%)
2019-12-10T22:26:46.210+05:30 [Info] Indexer CPU Utilization 0.0000
2019-12-10T22:26:46.210+05:30 [Info] Indexer IO Mean 0.0000
2019-12-10T22:26:46.210+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2019-12-10T22:26:46.210+05:30 [Info] Indexer Drain Rate Mean 0.0000
2019-12-10T22:26:46.210+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2019-12-10T22:26:46.210+05:30 [Info] Indexer Scan Rate Mean 0.0000
2019-12-10T22:26:46.210+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2019-12-10T22:26:46.210+05:30 [Info] Indexer Data Size Mean 1258291200 (1.17188G)
2019-12-10T22:26:46.210+05:30 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2019-12-10T22:26:46.210+05:30 [Info] Total Index Data (from non-deleted node) 0
2019-12-10T22:26:46.210+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2019-12-10T22:26:46.210+05:30 [Info] No. Index (from non-deleted node) 0
2019-12-10T22:26:46.210+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2019-12-10T22:26:46.210+05:30 [Info] ----------------------------------------
2019-12-10T22:26:46.210+05:30 [Info] Number of indexes: 1
2019-12-10T22:26:46.210+05:30 [Info] Number of indexers: 1
2019-12-10T22:26:46.210+05:30 [Info] Avg Index Size: 1258291200 (1.17188G)
2019-12-10T22:26:46.210+05:30 [Info] Max Index Size: 1258291200 (1.17188G)
2019-12-10T22:26:46.210+05:30 [Info] Max Indexer Overhead: 0 (0)
2019-12-10T22:26:46.210+05:30 [Info] Avg Index Cpu: 0.0000
2019-12-10T22:26:46.210+05:30 [Info] Max Index Cpu: 0.0000
2019-12-10T22:26:46.210+05:30 [Info] Num Estimation Run: 1
2019-12-10T22:26:46.210+05:30 [Info] ----------------------------------------
2019-12-10T22:26:46.210+05:30 [Info] Memory Quota 1572864000 (1.46484G)
2019-12-10T22:26:46.210+05:30 [Info] CPU Quota 4
2019-12-10T22:26:46.210+05:30 [Info] Max Cpu Utilization -1
2019-12-10T22:26:46.210+05:30 [Info] Max Memory Utilization -1
2019-12-10T22:26:46.210+05:30 [Info] ----------------------------------------
2019-12-10T22:26:46.210+05:30 [Info] Total iteration: 0
2019-12-10T22:26:46.210+05:30 [Info] RandomSwap time: 0ns
2019-12-10T22:26:46.210+05:30 [Info] RandomSwap call: 0
2019-12-10T22:26:46.210+05:30 [Info] RandomSwap iteration: 0
2019-12-10T22:26:46.210+05:30 [Info] RandomMove time: 0ns
2019-12-10T22:26:46.210+05:30 [Info] RandomMove call: 0 (empty index 0)
2019-12-10T22:26:46.211+05:30 [Info] ExhaustMove time: 0ns
2019-12-10T22:26:46.211+05:30 [Info] ExhaustMove call: 0
2019-12-10T22:26:46.211+05:30 [Info] ExhaustSwap time: 0ns
2019-12-10T22:26:46.211+05:30 [Info] ExhaustSwap call: 0
2019-12-10T22:26:46.211+05:30 [Info] ----------------------------------------
2019-12-10T22:26:46.211+05:30 [Info] 
2019-12-10T22:26:46.211+05:30 [Info] Indexer serverGroup:Group 1, nodeId:127.0.0.1:9001, nodeUUID:280ac5e1026b443deea667c0a12beceb, useLiveData:true
2019-12-10T22:26:46.211+05:30 [Info] Indexer total memory:1258291200 (1.17188G), mem:1258291200 (1.17188G), overhead:0 (0), min:1258291200 (1.17188G), data:1258291200 (1.17188G) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:1
2019-12-10T22:26:46.211+05:30 [Info] Indexer isDeleted:false isNew:false exclude: meetConstraint:true
2019-12-10T22:26:46.211+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:26:46.211+05:30 [Info] 		Index name:arridx_friends 0, bucket:default, defnId:2033787644074089091, instId:16380178195139616687, Partition: 0, new/moved:true
2019-12-10T22:26:46.211+05:30 [Info] 		Index total memory:1258291200 (1.17188G), mem:1258291200 (1.17188G), overhead:0 (0), min:1258291200 (1.17188G), data:1258291200 (1.17188G) cpu:0.0000 io:0 (0) scan:0 drain:0
2019-12-10T22:26:46.211+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:26:46.211+05:30 [Info] ****************************************
2019-12-10T22:26:46.211+05:30 [Info] send commit create request to watcher 127.0.0.1:9106
2019-12-10T22:26:46.466+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:26:46.466+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:26:46.471+05:30 [Info] switched currmeta from 1268 -> 1270 force true 
2019-12-10T22:26:46.472+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:26:46.472+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:26:46.475+05:30 [Info] switched currmeta from 1272 -> 1274 force true 
2019-12-10T22:26:46.633+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:26:46.633+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:26:46.638+05:30 [Info] switched currmeta from 1270 -> 1271 force true 
2019-12-10T22:26:46.643+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:26:46.643+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:26:46.645+05:30 [Info] switched currmeta from 1274 -> 1275 force true 
2019-12-10T22:26:46.731+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:26:46.731+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:26:46.737+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:26:46.737+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:26:46.737+05:30 [Info] switched currmeta from 1275 -> 1276 force true 
2019-12-10T22:26:46.740+05:30 [Info] switched currmeta from 1271 -> 1272 force true 
2019-12-10T22:26:46.903+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:26:46.903+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:26:46.904+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:26:46.904+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:26:46.907+05:30 [Info] switched currmeta from 1272 -> 1273 force true 
2019-12-10T22:26:46.909+05:30 [Info] switched currmeta from 1276 -> 1277 force true 
2019-12-10T22:26:49.684+05:30 [Info] CreateIndex 2033787644074089091 default/arridx_friends using:forestdb exprType:N1QL whereExpr:() secExprs:([(distinct (`friends`))]) desc:[] isPrimary:false scheme:SINGLE  partitionKeys:([]) with: - elapsed(4.116485815s) err()
2019/12/10 22:26:49 Created the secondary index arridx_friends. Waiting for it become active
2019-12-10T22:26:49.684+05:30 [Info] metadata provider version changed 1277 -> 1278
2019-12-10T22:26:49.684+05:30 [Info] switched currmeta from 1277 -> 1278 force false 
2019/12/10 22:26:49 Index is now active
2019-12-10T22:26:49.690+05:30 [Error] transport error between 127.0.0.1:40796->127.0.0.1:9107: write tcp 127.0.0.1:40796->127.0.0.1:9107: write: broken pipe
2019-12-10T22:26:49.690+05:30 [Error] [GsiScanClient:"127.0.0.1:9107"]  request transport failed `write tcp 127.0.0.1:40796->127.0.0.1:9107: write: broken pipe`
2019-12-10T22:26:49.690+05:30 [Warn] scan failed: requestId  queryport 127.0.0.1:9107 inst 18065756831487365417 partition [0]
2019-12-10T22:26:49.691+05:30 [Warn] Scan failed with error for index 2033787644074089091.  Trying scan again with replica, reqId: :  write tcp 127.0.0.1:40796->127.0.0.1:9107: write: broken pipe from [127.0.0.1:9107] ...
2019-12-10T22:26:49.691+05:30 [Error] PickRandom: Fail to find indexer for all index partitions. Num partition 1.  Partition with instances 0 
2019-12-10T22:26:49.691+05:30 [Warn] Fail to find indexers to satisfy query request.  Trying scan again for index 2033787644074089091, reqId: :  write tcp 127.0.0.1:40796->127.0.0.1:9107: write: broken pipe from [127.0.0.1:9107] ...
2019-12-10T22:26:49.690+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] closing unhealthy connection "127.0.0.1:40796"
2019-12-10T22:26:49.702+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] open new connection ...
2019-12-10T22:26:49.741+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:26:49.741+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:26:49.742+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:26:49.742+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:26:49.746+05:30 [Info] switched currmeta from 1278 -> 1278 force true 
2019-12-10T22:26:49.746+05:30 [Info] switched currmeta from 1273 -> 1274 force true 
2019/12/10 22:26:49 Expected and Actual scan responses are the same
2019/12/10 22:26:49 Expected and Actual scan responses are the same
--- PASS: TestRangeArrayIndex_Distinct (74.25s)
=== RUN   TestUpdateArrayIndex_Distinct
2019/12/10 22:26:49 In TestUpdateArrayIndex_Distinct()
2019/12/10 22:26:49 In DropAllSecondaryIndexes()
2019/12/10 22:26:49 Index found:  arridx_friends
2019-12-10T22:26:50.126+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:26:50.126+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:26:50.130+05:30 [Info] switched currmeta from 1274 -> 1275 force true 
2019-12-10T22:26:50.135+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:26:50.135+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:26:50.137+05:30 [Info] switched currmeta from 1278 -> 1279 force true 
2019-12-10T22:26:50.214+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:26:50.214+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:26:50.218+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:26:50.218+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:26:50.219+05:30 [Info] switched currmeta from 1275 -> 1275 force true 
2019-12-10T22:26:50.229+05:30 [Info] switched currmeta from 1279 -> 1279 force true 
2019-12-10T22:26:50.316+05:30 [Info] metadata provider version changed 1279 -> 1280
2019-12-10T22:26:50.316+05:30 [Info] switched currmeta from 1279 -> 1280 force false 
2019-12-10T22:26:50.316+05:30 [Info] DropIndex 2033787644074089091 - elapsed(333.888549ms), err()
2019/12/10 22:26:50 Dropped index arridx_friends
2019-12-10T22:26:50.446+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:26:50.449+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:26:50.449+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:26:50.453+05:30 [Info] switched currmeta from 1275 -> 1276 force true 
2019-12-10T22:26:50.456+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:26:50.456+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:26:50.473+05:30 [Info] switched currmeta from 1280 -> 1280 force true 
2019-12-10T22:26:50.535+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:26:50.535+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:26:50.545+05:30 [Info] switched currmeta from 1276 -> 1276 force true 
2019-12-10T22:26:50.557+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:26:50.557+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:26:50.563+05:30 [Info] switched currmeta from 1280 -> 1280 force true 
2019-12-10T22:26:50.599+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:26:50.674+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:26:50.674+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:26:50.676+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:26:50.676+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:26:50.679+05:30 [Info] switched currmeta from 1280 -> 1280 force true 
2019-12-10T22:26:50.684+05:30 [Info] switched currmeta from 1276 -> 1276 force true 
2019-12-10T22:26:52.564+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:26:52.633+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:26:52.633+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:26:52.633+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:26:52.633+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:26:52.637+05:30 [Info] switched currmeta from 1276 -> 1276 force true 
2019-12-10T22:26:52.637+05:30 [Info] switched currmeta from 1280 -> 1280 force true 
2019-12-10T22:26:55.584+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:26:55.617+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:26:55.617+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:26:55.621+05:30 [Info] switched currmeta from 1280 -> 1280 force true 
2019-12-10T22:26:55.623+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:26:55.623+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:26:55.626+05:30 [Info] switched currmeta from 1276 -> 1276 force true 
2019-12-10T22:26:55.866+05:30 [Info] connected with 1 indexers
2019-12-10T22:26:55.866+05:30 [Info] client load stats {}
2019-12-10T22:26:55.866+05:30 [Info] client hit stats {}
2019-12-10T22:26:55.866+05:30 [Info] client pending item stats {}
2019-12-10T22:26:55.866+05:30 [Info] client stats current {}
2019-12-10T22:26:55.873+05:30 [Info] num concurrent scans {0}
2019-12-10T22:26:55.874+05:30 [Info] average scan response {70 ms}
2019-12-10T22:26:58.387+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] active conns 0, free conns 1
2019-12-10T22:27:00.397+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:27:00.451+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:27:00.451+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:27:00.453+05:30 [Info] switched currmeta from 1276 -> 1276 force true 
2019-12-10T22:27:00.455+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:27:00.455+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:27:00.458+05:30 [Info] switched currmeta from 1280 -> 1280 force true 
2019-12-10T22:27:02.552+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:27:02.613+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:27:02.613+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:27:02.614+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:27:02.614+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:27:02.620+05:30 [Info] switched currmeta from 1280 -> 1280 force true 
2019-12-10T22:27:02.620+05:30 [Info] switched currmeta from 1276 -> 1276 force true 
2019-12-10T22:27:38.006+05:30 [Info] connected with 1 indexers
2019-12-10T22:27:38.006+05:30 [Info] client load stats {}
2019-12-10T22:27:38.006+05:30 [Info] client hit stats {}
2019-12-10T22:27:38.006+05:30 [Info] client pending item stats {}
2019-12-10T22:27:38.006+05:30 [Info] client stats current {}
2019-12-10T22:27:38.009+05:30 [Info] num concurrent scans {0}
2019-12-10T22:27:38.009+05:30 [Info] average scan response {65 ms}
2019-12-10T22:27:40.595+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] active conns 0, free conns 1
2019-12-10T22:27:55.866+05:30 [Info] connected with 1 indexers
2019-12-10T22:27:55.866+05:30 [Info] client load stats {}
2019-12-10T22:27:55.866+05:30 [Info] client hit stats {}
2019-12-10T22:27:55.866+05:30 [Info] client pending item stats {}
2019-12-10T22:27:55.866+05:30 [Info] client stats current {}
2019-12-10T22:27:55.874+05:30 [Info] num concurrent scans {0}
2019-12-10T22:27:55.874+05:30 [Info] average scan response {70 ms}
2019-12-10T22:27:58.404+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] active conns 0, free conns 1
2019/12/10 22:27:58 Flushed the bucket default
2019-12-10T22:27:59.746+05:30 [Info] send prepare create request to watcher 127.0.0.1:9106
2019-12-10T22:27:59.747+05:30 [Info] Indexer 127.0.0.1:9106 accept prepare request. Index (default, arridx_friends)
2019-12-10T22:27:59.809+05:30 [Info] Planner::processCreateToken: There is no create token to process for node 127.0.0.1:9001
2019-12-10T22:28:00.213+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node 127.0.0.1:9001
2019-12-10T22:28:00.312+05:30 [Info] ************ Indexer Layout *************
2019-12-10T22:28:00.312+05:30 [Info] Score: 0
2019-12-10T22:28:00.312+05:30 [Info] variation: 0
2019-12-10T22:28:00.312+05:30 [Info] ElapsedTime: 6ms
2019-12-10T22:28:00.312+05:30 [Info] ConvergenceTime: 2us
2019-12-10T22:28:00.312+05:30 [Info] Iteration: 0
2019-12-10T22:28:00.312+05:30 [Info] Move: 0
2019-12-10T22:28:00.312+05:30 [Info] ----------------------------------------
2019-12-10T22:28:00.312+05:30 [Info] Indexer Memory Mean 1258291200 (1.17188G)
2019-12-10T22:28:00.312+05:30 [Info] Indexer Memory Deviation 0 (0) (0.00%)
2019-12-10T22:28:00.312+05:30 [Info] Indexer Memory Utilization 0.8000
2019-12-10T22:28:00.312+05:30 [Info] Indexer CPU Mean 0.0000
2019-12-10T22:28:00.312+05:30 [Info] Indexer CPU Deviation 0.00 (0.00%)
2019-12-10T22:28:00.312+05:30 [Info] Indexer CPU Utilization 0.0000
2019-12-10T22:28:00.312+05:30 [Info] Indexer IO Mean 0.0000
2019-12-10T22:28:00.312+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2019-12-10T22:28:00.312+05:30 [Info] Indexer Drain Rate Mean 0.0000
2019-12-10T22:28:00.312+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2019-12-10T22:28:00.312+05:30 [Info] Indexer Scan Rate Mean 0.0000
2019-12-10T22:28:00.312+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2019-12-10T22:28:00.312+05:30 [Info] Indexer Data Size Mean 1258291200 (1.17188G)
2019-12-10T22:28:00.312+05:30 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2019-12-10T22:28:00.312+05:30 [Info] Total Index Data (from non-deleted node) 0
2019-12-10T22:28:00.312+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2019-12-10T22:28:00.312+05:30 [Info] No. Index (from non-deleted node) 0
2019-12-10T22:28:00.312+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2019-12-10T22:28:00.312+05:30 [Info] ----------------------------------------
2019-12-10T22:28:00.312+05:30 [Info] Number of indexes: 1
2019-12-10T22:28:00.312+05:30 [Info] Number of indexers: 1
2019-12-10T22:28:00.312+05:30 [Info] Avg Index Size: 1258291200 (1.17188G)
2019-12-10T22:28:00.312+05:30 [Info] Max Index Size: 1258291200 (1.17188G)
2019-12-10T22:28:00.312+05:30 [Info] Max Indexer Overhead: 0 (0)
2019-12-10T22:28:00.312+05:30 [Info] Avg Index Cpu: 0.0000
2019-12-10T22:28:00.312+05:30 [Info] Max Index Cpu: 0.0000
2019-12-10T22:28:00.312+05:30 [Info] Num Estimation Run: 1
2019-12-10T22:28:00.312+05:30 [Info] ----------------------------------------
2019-12-10T22:28:00.312+05:30 [Info] Memory Quota 1572864000 (1.46484G)
2019-12-10T22:28:00.312+05:30 [Info] CPU Quota 4
2019-12-10T22:28:00.312+05:30 [Info] Max Cpu Utilization -1
2019-12-10T22:28:00.312+05:30 [Info] Max Memory Utilization -1
2019-12-10T22:28:00.312+05:30 [Info] ----------------------------------------
2019-12-10T22:28:00.312+05:30 [Info] Total iteration: 0
2019-12-10T22:28:00.312+05:30 [Info] RandomSwap time: 0ns
2019-12-10T22:28:00.312+05:30 [Info] RandomSwap call: 0
2019-12-10T22:28:00.312+05:30 [Info] RandomSwap iteration: 0
2019-12-10T22:28:00.312+05:30 [Info] RandomMove time: 0ns
2019-12-10T22:28:00.312+05:30 [Info] RandomMove call: 0 (empty index 0)
2019-12-10T22:28:00.312+05:30 [Info] ExhaustMove time: 0ns
2019-12-10T22:28:00.312+05:30 [Info] ExhaustMove call: 0
2019-12-10T22:28:00.312+05:30 [Info] ExhaustSwap time: 0ns
2019-12-10T22:28:00.312+05:30 [Info] ExhaustSwap call: 0
2019-12-10T22:28:00.312+05:30 [Info] ----------------------------------------
2019-12-10T22:28:00.312+05:30 [Info] 
2019-12-10T22:28:00.312+05:30 [Info] Indexer serverGroup:Group 1, nodeId:127.0.0.1:9001, nodeUUID:280ac5e1026b443deea667c0a12beceb, useLiveData:true
2019-12-10T22:28:00.312+05:30 [Info] Indexer total memory:1258291200 (1.17188G), mem:1258291200 (1.17188G), overhead:0 (0), min:1258291200 (1.17188G), data:1258291200 (1.17188G) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:1
2019-12-10T22:28:00.312+05:30 [Info] Indexer isDeleted:false isNew:false exclude: meetConstraint:true
2019-12-10T22:28:00.312+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:28:00.312+05:30 [Info] 		Index name:arridx_friends 0, bucket:default, defnId:10171276462831183371, instId:11072986396863034092, Partition: 0, new/moved:true
2019-12-10T22:28:00.312+05:30 [Info] 		Index total memory:1258291200 (1.17188G), mem:1258291200 (1.17188G), overhead:0 (0), min:1258291200 (1.17188G), data:1258291200 (1.17188G) cpu:0.0000 io:0 (0) scan:0 drain:0
2019-12-10T22:28:00.312+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:28:00.312+05:30 [Info] ****************************************
2019-12-10T22:28:00.312+05:30 [Info] send commit create request to watcher 127.0.0.1:9106
2019-12-10T22:28:00.526+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:28:00.526+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:28:00.528+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:28:00.528+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:28:00.530+05:30 [Info] switched currmeta from 1280 -> 1282 force true 
2019-12-10T22:28:00.530+05:30 [Info] switched currmeta from 1276 -> 1278 force true 
2019-12-10T22:28:00.681+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:28:00.681+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:28:00.686+05:30 [Info] switched currmeta from 1278 -> 1279 force true 
2019-12-10T22:28:00.691+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:28:00.691+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:28:00.695+05:30 [Info] switched currmeta from 1282 -> 1283 force true 
2019-12-10T22:28:00.775+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:28:00.775+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:28:00.778+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:28:00.778+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:28:00.787+05:30 [Info] switched currmeta from 1279 -> 1280 force true 
2019-12-10T22:28:00.787+05:30 [Info] switched currmeta from 1283 -> 1284 force true 
2019-12-10T22:28:00.945+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:28:00.945+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:28:00.946+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:28:00.946+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:28:00.949+05:30 [Info] switched currmeta from 1284 -> 1285 force true 
2019-12-10T22:28:00.949+05:30 [Info] switched currmeta from 1280 -> 1281 force true 
2019-12-10T22:28:03.561+05:30 [Info] CreateIndex 10171276462831183371 default/arridx_friends using:forestdb exprType:N1QL whereExpr:() secExprs:([(distinct (`friends`))]) desc:[] isPrimary:false scheme:SINGLE  partitionKeys:([]) with: - elapsed(3.815722047s) err()
2019/12/10 22:28:03 Created the secondary index arridx_friends. Waiting for it become active
2019-12-10T22:28:03.561+05:30 [Info] metadata provider version changed 1285 -> 1286
2019-12-10T22:28:03.561+05:30 [Info] switched currmeta from 1285 -> 1286 force false 
2019/12/10 22:28:03 Index is now active
2019-12-10T22:28:03.630+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:28:03.630+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:28:03.630+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:28:03.630+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:28:03.635+05:30 [Info] switched currmeta from 1281 -> 1282 force true 
2019-12-10T22:28:03.636+05:30 [Info] switched currmeta from 1286 -> 1286 force true 
2019/12/10 22:28:03 Expected and Actual scan responses are the same
2019/12/10 22:28:04 Expected and Actual scan responses are the same
2019/12/10 22:28:05 Expected and Actual scan responses are the same
--- PASS: TestUpdateArrayIndex_Distinct (75.25s)
=== RUN   TestRangeArrayIndex_Duplicate
2019/12/10 22:28:05 In TestRangeArrayIndex_Duplicate()
2019/12/10 22:28:05 In DropAllSecondaryIndexes()
2019/12/10 22:28:05 Index found:  arridx_friends
2019-12-10T22:28:05.401+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:28:05.401+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:28:05.404+05:30 [Info] switched currmeta from 1282 -> 1283 force true 
2019-12-10T22:28:05.412+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:28:05.412+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:28:05.417+05:30 [Info] switched currmeta from 1286 -> 1287 force true 
2019-12-10T22:28:05.518+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:28:05.518+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:28:05.523+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:28:05.523+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:28:05.525+05:30 [Info] switched currmeta from 1287 -> 1287 force true 
2019-12-10T22:28:05.527+05:30 [Info] switched currmeta from 1283 -> 1283 force true 
2019-12-10T22:28:05.549+05:30 [Info] metadata provider version changed 1287 -> 1288
2019-12-10T22:28:05.549+05:30 [Info] switched currmeta from 1287 -> 1288 force false 
2019-12-10T22:28:05.549+05:30 [Info] DropIndex 10171276462831183371 - elapsed(320.210527ms), err()
2019/12/10 22:28:05 Dropped index arridx_friends
2019-12-10T22:28:05.712+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:28:05.724+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:28:05.724+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:28:05.730+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:28:05.730+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:28:05.743+05:30 [Info] switched currmeta from 1283 -> 1284 force true 
2019-12-10T22:28:05.749+05:30 [Info] switched currmeta from 1288 -> 1288 force true 
2019-12-10T22:28:05.846+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:28:05.846+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:28:05.851+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:28:05.851+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:28:05.861+05:30 [Info] switched currmeta from 1284 -> 1284 force true 
2019-12-10T22:28:05.863+05:30 [Info] switched currmeta from 1288 -> 1288 force true 
2019-12-10T22:28:07.555+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:28:07.613+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:28:07.613+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:28:07.619+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:28:07.619+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:28:07.621+05:30 [Info] switched currmeta from 1284 -> 1284 force true 
2019-12-10T22:28:07.626+05:30 [Info] switched currmeta from 1288 -> 1288 force true 
2019-12-10T22:28:10.399+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:28:10.451+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:28:10.451+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:28:10.453+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:28:10.453+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:28:10.456+05:30 [Info] switched currmeta from 1288 -> 1288 force true 
2019-12-10T22:28:10.457+05:30 [Info] switched currmeta from 1284 -> 1284 force true 
2019-12-10T22:28:10.604+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:28:10.634+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:28:10.634+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:28:10.636+05:30 [Info] switched currmeta from 1288 -> 1288 force true 
2019-12-10T22:28:10.637+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:28:10.637+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:28:10.640+05:30 [Info] switched currmeta from 1284 -> 1284 force true 
2019-12-10T22:28:15.400+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:28:15.446+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:28:15.446+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:28:15.452+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:28:15.452+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:28:15.452+05:30 [Info] switched currmeta from 1284 -> 1284 force true 
2019-12-10T22:28:15.454+05:30 [Info] switched currmeta from 1288 -> 1288 force true 
2019-12-10T22:28:17.553+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:28:17.595+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:28:17.596+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:28:17.600+05:30 [Info] switched currmeta from 1288 -> 1288 force true 
2019-12-10T22:28:17.604+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:28:17.604+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:28:17.606+05:30 [Info] switched currmeta from 1284 -> 1284 force true 
2019-12-10T22:28:38.006+05:30 [Info] connected with 1 indexers
2019-12-10T22:28:38.006+05:30 [Info] client load stats {}
2019-12-10T22:28:38.006+05:30 [Info] client hit stats {}
2019-12-10T22:28:38.006+05:30 [Info] client pending item stats {}
2019-12-10T22:28:38.006+05:30 [Info] client stats current {}
2019-12-10T22:28:38.009+05:30 [Info] num concurrent scans {0}
2019-12-10T22:28:38.009+05:30 [Info] average scan response {65 ms}
2019-12-10T22:28:40.614+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] active conns 0, free conns 1
2019-12-10T22:28:55.866+05:30 [Info] connected with 1 indexers
2019-12-10T22:28:55.866+05:30 [Info] client load stats {}
2019-12-10T22:28:55.866+05:30 [Info] client hit stats {}
2019-12-10T22:28:55.866+05:30 [Info] client pending item stats {}
2019-12-10T22:28:55.866+05:30 [Info] client stats current {}
2019-12-10T22:28:55.874+05:30 [Info] num concurrent scans {0}
2019-12-10T22:28:55.874+05:30 [Info] average scan response {150 ms}
2019-12-10T22:28:58.424+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] active conns 0, free conns 1
2019/12/10 22:29:12 Flushed the bucket default
2019-12-10T22:29:13.756+05:30 [Info] send prepare create request to watcher 127.0.0.1:9106
2019-12-10T22:29:13.757+05:30 [Info] Indexer 127.0.0.1:9106 accept prepare request. Index (default, arridx_friends)
2019-12-10T22:29:13.804+05:30 [Info] Planner::processCreateToken: There is no create token to process for node 127.0.0.1:9001
2019-12-10T22:29:14.185+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node 127.0.0.1:9001
2019-12-10T22:29:14.261+05:30 [Info] ************ Indexer Layout *************
2019-12-10T22:29:14.261+05:30 [Info] Score: 0
2019-12-10T22:29:14.262+05:30 [Info] variation: 0
2019-12-10T22:29:14.262+05:30 [Info] ElapsedTime: 6ms
2019-12-10T22:29:14.262+05:30 [Info] ConvergenceTime: 2us
2019-12-10T22:29:14.262+05:30 [Info] Iteration: 0
2019-12-10T22:29:14.262+05:30 [Info] Move: 0
2019-12-10T22:29:14.262+05:30 [Info] ----------------------------------------
2019-12-10T22:29:14.262+05:30 [Info] Indexer Memory Mean 1258291200 (1.17188G)
2019-12-10T22:29:14.262+05:30 [Info] Indexer Memory Deviation 0 (0) (0.00%)
2019-12-10T22:29:14.262+05:30 [Info] Indexer Memory Utilization 0.8000
2019-12-10T22:29:14.262+05:30 [Info] Indexer CPU Mean 0.0000
2019-12-10T22:29:14.262+05:30 [Info] Indexer CPU Deviation 0.00 (0.00%)
2019-12-10T22:29:14.262+05:30 [Info] Indexer CPU Utilization 0.0000
2019-12-10T22:29:14.262+05:30 [Info] Indexer IO Mean 0.0000
2019-12-10T22:29:14.262+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2019-12-10T22:29:14.262+05:30 [Info] Indexer Drain Rate Mean 0.0000
2019-12-10T22:29:14.262+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2019-12-10T22:29:14.262+05:30 [Info] Indexer Scan Rate Mean 0.0000
2019-12-10T22:29:14.262+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2019-12-10T22:29:14.262+05:30 [Info] Indexer Data Size Mean 1258291200 (1.17188G)
2019-12-10T22:29:14.262+05:30 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2019-12-10T22:29:14.262+05:30 [Info] Total Index Data (from non-deleted node) 0
2019-12-10T22:29:14.262+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2019-12-10T22:29:14.262+05:30 [Info] No. Index (from non-deleted node) 0
2019-12-10T22:29:14.262+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2019-12-10T22:29:14.262+05:30 [Info] ----------------------------------------
2019-12-10T22:29:14.262+05:30 [Info] Number of indexes: 1
2019-12-10T22:29:14.262+05:30 [Info] Number of indexers: 1
2019-12-10T22:29:14.262+05:30 [Info] Avg Index Size: 1258291200 (1.17188G)
2019-12-10T22:29:14.262+05:30 [Info] Max Index Size: 1258291200 (1.17188G)
2019-12-10T22:29:14.262+05:30 [Info] Max Indexer Overhead: 0 (0)
2019-12-10T22:29:14.262+05:30 [Info] Avg Index Cpu: 0.0000
2019-12-10T22:29:14.262+05:30 [Info] Max Index Cpu: 0.0000
2019-12-10T22:29:14.262+05:30 [Info] Num Estimation Run: 1
2019-12-10T22:29:14.262+05:30 [Info] ----------------------------------------
2019-12-10T22:29:14.262+05:30 [Info] Memory Quota 1572864000 (1.46484G)
2019-12-10T22:29:14.262+05:30 [Info] CPU Quota 4
2019-12-10T22:29:14.262+05:30 [Info] Max Cpu Utilization -1
2019-12-10T22:29:14.262+05:30 [Info] Max Memory Utilization -1
2019-12-10T22:29:14.262+05:30 [Info] ----------------------------------------
2019-12-10T22:29:14.262+05:30 [Info] Total iteration: 0
2019-12-10T22:29:14.262+05:30 [Info] RandomSwap time: 0ns
2019-12-10T22:29:14.262+05:30 [Info] RandomSwap call: 0
2019-12-10T22:29:14.262+05:30 [Info] RandomSwap iteration: 0
2019-12-10T22:29:14.263+05:30 [Info] RandomMove time: 0ns
2019-12-10T22:29:14.263+05:30 [Info] RandomMove call: 0 (empty index 0)
2019-12-10T22:29:14.263+05:30 [Info] ExhaustMove time: 0ns
2019-12-10T22:29:14.263+05:30 [Info] ExhaustMove call: 0
2019-12-10T22:29:14.263+05:30 [Info] ExhaustSwap time: 0ns
2019-12-10T22:29:14.263+05:30 [Info] ExhaustSwap call: 0
2019-12-10T22:29:14.263+05:30 [Info] ----------------------------------------
2019-12-10T22:29:14.263+05:30 [Info] 
2019-12-10T22:29:14.263+05:30 [Info] Indexer serverGroup:Group 1, nodeId:127.0.0.1:9001, nodeUUID:280ac5e1026b443deea667c0a12beceb, useLiveData:true
2019-12-10T22:29:14.263+05:30 [Info] Indexer total memory:1258291200 (1.17188G), mem:1258291200 (1.17188G), overhead:0 (0), min:1258291200 (1.17188G), data:1258291200 (1.17188G) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:1
2019-12-10T22:29:14.263+05:30 [Info] Indexer isDeleted:false isNew:false exclude: meetConstraint:true
2019-12-10T22:29:14.263+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:29:14.263+05:30 [Info] 		Index name:arridx_friends 0, bucket:default, defnId:9653170433918775397, instId:7174879325104669282, Partition: 0, new/moved:true
2019-12-10T22:29:14.263+05:30 [Info] 		Index total memory:1258291200 (1.17188G), mem:1258291200 (1.17188G), overhead:0 (0), min:1258291200 (1.17188G), data:1258291200 (1.17188G) cpu:0.0000 io:0 (0) scan:0 drain:0
2019-12-10T22:29:14.263+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:29:14.263+05:30 [Info] ****************************************
2019-12-10T22:29:14.263+05:30 [Info] send commit create request to watcher 127.0.0.1:9106
2019-12-10T22:29:14.489+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:29:14.489+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:29:14.492+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:29:14.492+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:29:14.494+05:30 [Info] switched currmeta from 1288 -> 1290 force true 
2019-12-10T22:29:14.495+05:30 [Info] switched currmeta from 1284 -> 1286 force true 
2019-12-10T22:29:14.634+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:29:14.634+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:29:14.637+05:30 [Info] switched currmeta from 1290 -> 1291 force true 
2019-12-10T22:29:14.643+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:29:14.643+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:29:14.647+05:30 [Info] switched currmeta from 1286 -> 1287 force true 
2019-12-10T22:29:14.691+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:29:14.691+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:29:14.694+05:30 [Info] switched currmeta from 1291 -> 1292 force true 
2019-12-10T22:29:14.707+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:29:14.707+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:29:14.709+05:30 [Info] switched currmeta from 1287 -> 1288 force true 
2019-12-10T22:29:14.893+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:29:14.893+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:29:14.899+05:30 [Info] switched currmeta from 1292 -> 1293 force true 
2019-12-10T22:29:14.920+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:29:14.920+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:29:14.924+05:30 [Info] switched currmeta from 1288 -> 1289 force true 
2019-12-10T22:29:17.467+05:30 [Info] CreateIndex 9653170433918775397 default/arridx_friends using:forestdb exprType:N1QL whereExpr:() secExprs:([(all (`friends`))]) desc:[] isPrimary:false scheme:SINGLE  partitionKeys:([]) with: - elapsed(3.712178838s) err()
2019/12/10 22:29:17 Created the secondary index arridx_friends. Waiting for it become active
2019-12-10T22:29:17.467+05:30 [Info] metadata provider version changed 1293 -> 1294
2019-12-10T22:29:17.467+05:30 [Info] switched currmeta from 1293 -> 1294 force false 
2019/12/10 22:29:17 Index is now active
2019-12-10T22:29:17.530+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:29:17.530+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:29:17.532+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:29:17.532+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:29:17.534+05:30 [Info] switched currmeta from 1294 -> 1294 force true 
2019-12-10T22:29:17.535+05:30 [Info] switched currmeta from 1289 -> 1290 force true 
2019/12/10 22:29:17 Expected and Actual scan responses are the same
2019/12/10 22:29:17 Expected and Actual scan responses are the same
--- PASS: TestRangeArrayIndex_Duplicate (72.72s)
=== RUN   TestUpdateArrayIndex_Duplicate
2019/12/10 22:29:17 In TestUpdateArrayIndex_Duplicate()
2019/12/10 22:29:17 In DropAllSecondaryIndexes()
2019/12/10 22:29:17 Index found:  arridx_friends
2019-12-10T22:29:18.077+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:29:18.077+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:29:18.081+05:30 [Info] switched currmeta from 1294 -> 1295 force true 
2019-12-10T22:29:18.094+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:29:18.094+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:29:18.103+05:30 [Info] switched currmeta from 1290 -> 1291 force true 
2019-12-10T22:29:18.165+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:29:18.165+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:29:18.174+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:29:18.174+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:29:18.176+05:30 [Info] switched currmeta from 1295 -> 1295 force true 
2019-12-10T22:29:18.197+05:30 [Info] switched currmeta from 1291 -> 1291 force true 
2019-12-10T22:29:18.239+05:30 [Info] metadata provider version changed 1295 -> 1296
2019-12-10T22:29:18.239+05:30 [Info] switched currmeta from 1295 -> 1296 force false 
2019-12-10T22:29:18.239+05:30 [Info] DropIndex 9653170433918775397 - elapsed(292.094608ms), err()
2019/12/10 22:29:18 Dropped index arridx_friends
2019-12-10T22:29:18.405+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:29:18.405+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:29:18.406+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:29:18.406+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:29:18.416+05:30 [Info] switched currmeta from 1296 -> 1296 force true 
2019-12-10T22:29:18.441+05:30 [Info] switched currmeta from 1291 -> 1292 force true 
2019-12-10T22:29:20.405+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:29:20.456+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:29:20.456+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:29:20.459+05:30 [Info] switched currmeta from 1292 -> 1292 force true 
2019-12-10T22:29:20.463+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:29:20.463+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:29:20.465+05:30 [Info] switched currmeta from 1296 -> 1296 force true 
2019-12-10T22:29:20.623+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:29:20.660+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:29:20.660+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:29:20.661+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:29:20.661+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:29:20.663+05:30 [Info] switched currmeta from 1292 -> 1292 force true 
2019-12-10T22:29:20.663+05:30 [Info] switched currmeta from 1296 -> 1296 force true 
2019-12-10T22:29:22.571+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:29:22.616+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:29:22.616+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:29:22.621+05:30 [Info] switched currmeta from 1292 -> 1292 force true 
2019-12-10T22:29:22.626+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:29:22.626+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:29:22.628+05:30 [Info] switched currmeta from 1296 -> 1296 force true 
2019-12-10T22:29:25.626+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:29:25.666+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:29:25.666+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:29:25.670+05:30 [Info] switched currmeta from 1296 -> 1296 force true 
2019-12-10T22:29:25.675+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:29:25.675+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:29:25.681+05:30 [Info] switched currmeta from 1292 -> 1292 force true 
2019-12-10T22:29:27.555+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:29:27.591+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:29:27.591+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:29:27.600+05:30 [Info] switched currmeta from 1292 -> 1292 force true 
2019-12-10T22:29:27.605+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:29:27.605+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:29:27.607+05:30 [Info] switched currmeta from 1296 -> 1296 force true 
2019-12-10T22:29:30.399+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:29:30.449+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:29:30.449+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:29:30.455+05:30 [Info] switched currmeta from 1292 -> 1292 force true 
2019-12-10T22:29:30.456+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:29:30.456+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:29:30.465+05:30 [Info] switched currmeta from 1296 -> 1296 force true 
2019-12-10T22:29:38.006+05:30 [Info] connected with 1 indexers
2019-12-10T22:29:38.006+05:30 [Info] client load stats {}
2019-12-10T22:29:38.006+05:30 [Info] client hit stats {}
2019-12-10T22:29:38.006+05:30 [Info] client pending item stats {}
2019-12-10T22:29:38.006+05:30 [Info] client stats current {}
2019-12-10T22:29:38.009+05:30 [Info] num concurrent scans {0}
2019-12-10T22:29:38.009+05:30 [Info] average scan response {65 ms}
2019-12-10T22:29:40.634+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] active conns 0, free conns 1
2019-12-10T22:29:55.866+05:30 [Info] connected with 1 indexers
2019-12-10T22:29:55.866+05:30 [Info] client load stats {}
2019-12-10T22:29:55.866+05:30 [Info] client hit stats {}
2019-12-10T22:29:55.866+05:30 [Info] client pending item stats {}
2019-12-10T22:29:55.866+05:30 [Info] client stats current {}
2019-12-10T22:29:55.874+05:30 [Info] num concurrent scans {0}
2019-12-10T22:29:55.874+05:30 [Info] average scan response {178 ms}
2019-12-10T22:29:58.442+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] active conns 0, free conns 1
2019/12/10 22:30:26 Flushed the bucket default
2019-12-10T22:30:27.594+05:30 [Info] send prepare create request to watcher 127.0.0.1:9106
2019-12-10T22:30:27.595+05:30 [Info] Indexer 127.0.0.1:9106 accept prepare request. Index (default, arridx_friends)
2019-12-10T22:30:27.679+05:30 [Info] Planner::processCreateToken: There is no create token to process for node 127.0.0.1:9001
2019-12-10T22:30:28.098+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node 127.0.0.1:9001
2019-12-10T22:30:28.187+05:30 [Info] ************ Indexer Layout *************
2019-12-10T22:30:28.187+05:30 [Info] Score: 0
2019-12-10T22:30:28.187+05:30 [Info] variation: 0
2019-12-10T22:30:28.187+05:30 [Info] ElapsedTime: 6ms
2019-12-10T22:30:28.187+05:30 [Info] ConvergenceTime: 3us
2019-12-10T22:30:28.187+05:30 [Info] Iteration: 0
2019-12-10T22:30:28.187+05:30 [Info] Move: 0
2019-12-10T22:30:28.187+05:30 [Info] ----------------------------------------
2019-12-10T22:30:28.187+05:30 [Info] Indexer Memory Mean 1258291200 (1.17188G)
2019-12-10T22:30:28.187+05:30 [Info] Indexer Memory Deviation 0 (0) (0.00%)
2019-12-10T22:30:28.187+05:30 [Info] Indexer Memory Utilization 0.8000
2019-12-10T22:30:28.187+05:30 [Info] Indexer CPU Mean 0.0000
2019-12-10T22:30:28.187+05:30 [Info] Indexer CPU Deviation 0.00 (0.00%)
2019-12-10T22:30:28.187+05:30 [Info] Indexer CPU Utilization 0.0000
2019-12-10T22:30:28.187+05:30 [Info] Indexer IO Mean 0.0000
2019-12-10T22:30:28.187+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2019-12-10T22:30:28.187+05:30 [Info] Indexer Drain Rate Mean 0.0000
2019-12-10T22:30:28.187+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2019-12-10T22:30:28.187+05:30 [Info] Indexer Scan Rate Mean 0.0000
2019-12-10T22:30:28.187+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2019-12-10T22:30:28.187+05:30 [Info] Indexer Data Size Mean 1258291200 (1.17188G)
2019-12-10T22:30:28.187+05:30 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2019-12-10T22:30:28.187+05:30 [Info] Total Index Data (from non-deleted node) 0
2019-12-10T22:30:28.187+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2019-12-10T22:30:28.187+05:30 [Info] No. Index (from non-deleted node) 0
2019-12-10T22:30:28.187+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2019-12-10T22:30:28.187+05:30 [Info] ----------------------------------------
2019-12-10T22:30:28.187+05:30 [Info] Number of indexes: 1
2019-12-10T22:30:28.187+05:30 [Info] Number of indexers: 1
2019-12-10T22:30:28.187+05:30 [Info] Avg Index Size: 1258291200 (1.17188G)
2019-12-10T22:30:28.187+05:30 [Info] Max Index Size: 1258291200 (1.17188G)
2019-12-10T22:30:28.187+05:30 [Info] Max Indexer Overhead: 0 (0)
2019-12-10T22:30:28.187+05:30 [Info] Avg Index Cpu: 0.0000
2019-12-10T22:30:28.187+05:30 [Info] Max Index Cpu: 0.0000
2019-12-10T22:30:28.187+05:30 [Info] Num Estimation Run: 1
2019-12-10T22:30:28.187+05:30 [Info] ----------------------------------------
2019-12-10T22:30:28.187+05:30 [Info] Memory Quota 1572864000 (1.46484G)
2019-12-10T22:30:28.187+05:30 [Info] CPU Quota 4
2019-12-10T22:30:28.187+05:30 [Info] Max Cpu Utilization -1
2019-12-10T22:30:28.187+05:30 [Info] Max Memory Utilization -1
2019-12-10T22:30:28.187+05:30 [Info] ----------------------------------------
2019-12-10T22:30:28.187+05:30 [Info] Total iteration: 0
2019-12-10T22:30:28.187+05:30 [Info] RandomSwap time: 0ns
2019-12-10T22:30:28.187+05:30 [Info] RandomSwap call: 0
2019-12-10T22:30:28.187+05:30 [Info] RandomSwap iteration: 0
2019-12-10T22:30:28.187+05:30 [Info] RandomMove time: 0ns
2019-12-10T22:30:28.187+05:30 [Info] RandomMove call: 0 (empty index 0)
2019-12-10T22:30:28.187+05:30 [Info] ExhaustMove time: 0ns
2019-12-10T22:30:28.187+05:30 [Info] ExhaustMove call: 0
2019-12-10T22:30:28.187+05:30 [Info] ExhaustSwap time: 0ns
2019-12-10T22:30:28.187+05:30 [Info] ExhaustSwap call: 0
2019-12-10T22:30:28.187+05:30 [Info] ----------------------------------------
2019-12-10T22:30:28.187+05:30 [Info] 
2019-12-10T22:30:28.187+05:30 [Info] Indexer serverGroup:Group 1, nodeId:127.0.0.1:9001, nodeUUID:280ac5e1026b443deea667c0a12beceb, useLiveData:true
2019-12-10T22:30:28.187+05:30 [Info] Indexer total memory:1258291200 (1.17188G), mem:1258291200 (1.17188G), overhead:0 (0), min:1258291200 (1.17188G), data:1258291200 (1.17188G) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:1
2019-12-10T22:30:28.187+05:30 [Info] Indexer isDeleted:false isNew:false exclude: meetConstraint:true
2019-12-10T22:30:28.187+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:30:28.187+05:30 [Info] 		Index name:arridx_friends 0, bucket:default, defnId:11726076518073797138, instId:2859675336467132288, Partition: 0, new/moved:true
2019-12-10T22:30:28.187+05:30 [Info] 		Index total memory:1258291200 (1.17188G), mem:1258291200 (1.17188G), overhead:0 (0), min:1258291200 (1.17188G), data:1258291200 (1.17188G) cpu:0.0000 io:0 (0) scan:0 drain:0
2019-12-10T22:30:28.188+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:30:28.188+05:30 [Info] ****************************************
2019-12-10T22:30:28.188+05:30 [Info] send commit create request to watcher 127.0.0.1:9106
2019-12-10T22:30:28.379+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:28.379+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:28.385+05:30 [Info] switched currmeta from 1296 -> 1298 force true 
2019-12-10T22:30:28.390+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:28.390+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:28.392+05:30 [Info] switched currmeta from 1292 -> 1294 force true 
2019-12-10T22:30:28.531+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:28.531+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:28.531+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:28.531+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:28.541+05:30 [Info] switched currmeta from 1298 -> 1299 force true 
2019-12-10T22:30:28.541+05:30 [Info] switched currmeta from 1294 -> 1295 force true 
2019-12-10T22:30:28.628+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:28.628+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:28.629+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:28.629+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:28.634+05:30 [Info] switched currmeta from 1295 -> 1296 force true 
2019-12-10T22:30:28.634+05:30 [Info] switched currmeta from 1299 -> 1300 force true 
2019-12-10T22:30:28.834+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:28.834+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:28.834+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:28.834+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:28.836+05:30 [Info] switched currmeta from 1296 -> 1297 force true 
2019-12-10T22:30:28.837+05:30 [Info] switched currmeta from 1300 -> 1301 force true 
2019-12-10T22:30:31.591+05:30 [Info] CreateIndex 11726076518073797138 default/arridx_friends using:forestdb exprType:N1QL whereExpr:() secExprs:([(all (`friends`))]) desc:[] isPrimary:false scheme:SINGLE  partitionKeys:([]) with: - elapsed(3.996827336s) err()
2019/12/10 22:30:31 Created the secondary index arridx_friends. Waiting for it become active
2019-12-10T22:30:31.591+05:30 [Info] metadata provider version changed 1301 -> 1302
2019-12-10T22:30:31.591+05:30 [Info] switched currmeta from 1301 -> 1302 force false 
2019/12/10 22:30:31 Index is now active
2019-12-10T22:30:31.652+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:31.653+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:31.657+05:30 [Info] switched currmeta from 1302 -> 1302 force true 
2019-12-10T22:30:31.661+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:31.661+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:31.663+05:30 [Info] switched currmeta from 1297 -> 1298 force true 
2019/12/10 22:30:31 Expected and Actual scan responses are the same
2019-12-10T22:30:32.654+05:30 [Info] Rollback time has changed for index inst 17505973891200266198. New rollback time 1575996807857833855
2019-12-10T22:30:32.654+05:30 [Info] Rollback time has changed for index inst 17505973891200266198. New rollback time 1575996807857833855
2019/12/10 22:30:33 Expected and Actual scan responses are the same
2019/12/10 22:30:33 Expected and Actual scan responses are the same
--- PASS: TestUpdateArrayIndex_Duplicate (75.65s)
=== RUN   TestArrayIndexCornerCases
2019/12/10 22:30:33 In TestArrayIndexCornerCases()
2019-12-10T22:30:33.594+05:30 [Info] send prepare create request to watcher 127.0.0.1:9106
2019-12-10T22:30:33.595+05:30 [Info] Indexer 127.0.0.1:9106 accept prepare request. Index (default, arr_single)
2019-12-10T22:30:33.660+05:30 [Info] Planner::processCreateToken: There is no create token to process for node 127.0.0.1:9001
2019-12-10T22:30:34.078+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node 127.0.0.1:9001
2019-12-10T22:30:34.172+05:30 [Info] ************ Indexer Layout *************
2019-12-10T22:30:34.172+05:30 [Info] Score: 0
2019-12-10T22:30:34.172+05:30 [Info] variation: 0
2019-12-10T22:30:34.172+05:30 [Info] ElapsedTime: 8ms
2019-12-10T22:30:34.172+05:30 [Info] ConvergenceTime: 2us
2019-12-10T22:30:34.172+05:30 [Info] Iteration: 0
2019-12-10T22:30:34.172+05:30 [Info] Move: 0
2019-12-10T22:30:34.172+05:30 [Info] ----------------------------------------
2019-12-10T22:30:34.172+05:30 [Info] Indexer Memory Mean 1258291200 (1.17188G)
2019-12-10T22:30:34.172+05:30 [Info] Indexer Memory Deviation 0 (0) (0.00%)
2019-12-10T22:30:34.172+05:30 [Info] Indexer Memory Utilization 0.8000
2019-12-10T22:30:34.173+05:30 [Info] Indexer CPU Mean 0.0000
2019-12-10T22:30:34.173+05:30 [Info] Indexer CPU Deviation 0.00 (0.00%)
2019-12-10T22:30:34.173+05:30 [Info] Indexer CPU Utilization 0.0000
2019-12-10T22:30:34.173+05:30 [Info] Indexer IO Mean 0.0000
2019-12-10T22:30:34.173+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2019-12-10T22:30:34.173+05:30 [Info] Indexer Drain Rate Mean 0.0000
2019-12-10T22:30:34.173+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2019-12-10T22:30:34.173+05:30 [Info] Indexer Scan Rate Mean 0.0000
2019-12-10T22:30:34.173+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2019-12-10T22:30:34.173+05:30 [Info] Indexer Data Size Mean 1258952245 (1.17249G)
2019-12-10T22:30:34.173+05:30 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2019-12-10T22:30:34.173+05:30 [Info] Total Index Data (from non-deleted node) 0
2019-12-10T22:30:34.173+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2019-12-10T22:30:34.173+05:30 [Info] No. Index (from non-deleted node) 0
2019-12-10T22:30:34.173+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2019-12-10T22:30:34.173+05:30 [Info] ----------------------------------------
2019-12-10T22:30:34.173+05:30 [Info] Number of indexes: 2
2019-12-10T22:30:34.173+05:30 [Info] Number of indexers: 1
2019-12-10T22:30:34.173+05:30 [Info] Avg Index Size: 629145600 (600M)
2019-12-10T22:30:34.173+05:30 [Info] Max Index Size: 1258291200 (1.17188G)
2019-12-10T22:30:34.173+05:30 [Info] Max Indexer Overhead: 0 (0)
2019-12-10T22:30:34.173+05:30 [Info] Avg Index Cpu: 0.0000
2019-12-10T22:30:34.173+05:30 [Info] Max Index Cpu: 0.0000
2019-12-10T22:30:34.173+05:30 [Info] Num Estimation Run: 1
2019-12-10T22:30:34.173+05:30 [Info] ----------------------------------------
2019-12-10T22:30:34.173+05:30 [Info] Memory Quota 1572864000 (1.46484G)
2019-12-10T22:30:34.173+05:30 [Info] CPU Quota 4
2019-12-10T22:30:34.173+05:30 [Info] Max Cpu Utilization -1
2019-12-10T22:30:34.173+05:30 [Info] Max Memory Utilization -1
2019-12-10T22:30:34.173+05:30 [Info] ----------------------------------------
2019-12-10T22:30:34.173+05:30 [Info] Total iteration: 0
2019-12-10T22:30:34.173+05:30 [Info] RandomSwap time: 0ns
2019-12-10T22:30:34.173+05:30 [Info] RandomSwap call: 0
2019-12-10T22:30:34.173+05:30 [Info] RandomSwap iteration: 0
2019-12-10T22:30:34.173+05:30 [Info] RandomMove time: 0ns
2019-12-10T22:30:34.173+05:30 [Info] RandomMove call: 0 (empty index 0)
2019-12-10T22:30:34.173+05:30 [Info] ExhaustMove time: 0ns
2019-12-10T22:30:34.173+05:30 [Info] ExhaustMove call: 0
2019-12-10T22:30:34.173+05:30 [Info] ExhaustSwap time: 0ns
2019-12-10T22:30:34.173+05:30 [Info] ExhaustSwap call: 0
2019-12-10T22:30:34.173+05:30 [Info] ----------------------------------------
2019-12-10T22:30:34.173+05:30 [Info] 
2019-12-10T22:30:34.173+05:30 [Info] Indexer serverGroup:Group 1, nodeId:127.0.0.1:9001, nodeUUID:280ac5e1026b443deea667c0a12beceb, useLiveData:true
2019-12-10T22:30:34.173+05:30 [Info] Indexer total memory:1258291200 (1.17188G), mem:1258291200 (1.17188G), overhead:0 (0), min:1258423409 (1.172G), data:1258952245 (1.17249G) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:2
2019-12-10T22:30:34.173+05:30 [Info] Indexer isDeleted:false isNew:false exclude: meetConstraint:true
2019-12-10T22:30:34.173+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:30:34.173+05:30 [Info] 		Index name:arridx_friends 0, bucket:default, defnId:11726076518073797138, instId:17505973891200266198, Partition: 0, new/moved:false
2019-12-10T22:30:34.173+05:30 [Info] 		Index total memory:0 (0), mem:0 (0), overhead:0 (0), min:132209 (129.11K), data:661045 (645.552K) cpu:0.0000 io:0 (0) scan:0 drain:0
2019-12-10T22:30:34.173+05:30 [Info] 		Index resident:0% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:30:34.173+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:30:34.173+05:30 [Info] 		Index name:arr_single 0, bucket:default, defnId:4943923536053865142, instId:11626343829695265192, Partition: 0, new/moved:true
2019-12-10T22:30:34.173+05:30 [Info] 		Index total memory:1258291200 (1.17188G), mem:1258291200 (1.17188G), overhead:0 (0), min:1258291200 (1.17188G), data:1258291200 (1.17188G) cpu:0.0000 io:0 (0) scan:0 drain:0
2019-12-10T22:30:34.173+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:30:34.173+05:30 [Info] ****************************************
2019-12-10T22:30:34.174+05:30 [Info] send commit create request to watcher 127.0.0.1:9106
2019-12-10T22:30:34.391+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:34.391+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:34.397+05:30 [Info] switched currmeta from 1302 -> 1304 force true 
2019-12-10T22:30:34.397+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:34.397+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:34.405+05:30 [Info] switched currmeta from 1298 -> 1300 force true 
2019-12-10T22:30:34.552+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:34.552+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:34.558+05:30 [Info] switched currmeta from 1304 -> 1305 force true 
2019-12-10T22:30:34.567+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:34.567+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:34.569+05:30 [Info] switched currmeta from 1300 -> 1301 force true 
2019-12-10T22:30:34.644+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:34.644+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:34.644+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:34.644+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:34.647+05:30 [Info] switched currmeta from 1301 -> 1302 force true 
2019-12-10T22:30:34.649+05:30 [Info] switched currmeta from 1305 -> 1306 force true 
2019-12-10T22:30:34.831+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:34.831+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:34.831+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:34.831+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:34.841+05:30 [Info] switched currmeta from 1302 -> 1303 force true 
2019-12-10T22:30:34.841+05:30 [Info] switched currmeta from 1306 -> 1307 force true 
2019-12-10T22:30:35.643+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:30:35.710+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:35.710+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:35.712+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:35.712+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:35.718+05:30 [Info] switched currmeta from 1303 -> 1303 force true 
2019-12-10T22:30:35.719+05:30 [Info] switched currmeta from 1307 -> 1307 force true 
2019-12-10T22:30:37.623+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:37.623+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:37.627+05:30 [Info] switched currmeta from 1303 -> 1304 force true 
2019-12-10T22:30:37.627+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:37.627+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:37.631+05:30 [Info] switched currmeta from 1307 -> 1308 force true 
2019-12-10T22:30:38.006+05:30 [Info] connected with 1 indexers
2019-12-10T22:30:38.006+05:30 [Info] index 4943923536053865142 has replicas [14454778974833588294]: 
2019-12-10T22:30:38.006+05:30 [Info] index 11726076518073797138 has replicas [17505973891200266198]: 
2019-12-10T22:30:38.006+05:30 [Info] index 4943923536053865142 has equivalents [4943923536053865142]: 
2019-12-10T22:30:38.006+05:30 [Info] index 11726076518073797138 has equivalents [11726076518073797138]: 
2019-12-10T22:30:38.006+05:30 [Info] client load stats {"14454778974833588294": 0,"17505973891200266198": 0}
2019-12-10T22:30:38.006+05:30 [Info] client hit stats {"14454778974833588294": 0,"17505973891200266198": 0}
2019-12-10T22:30:38.006+05:30 [Info] client pending item stats {"17505973891200266198": 9223372036854775807,"14454778974833588294": 9223372036854775807}
2019-12-10T22:30:38.006+05:30 [Info] client stats current {"14454778974833588294": true,"17505973891200266198": true}
2019-12-10T22:30:38.009+05:30 [Info] num concurrent scans {0}
2019-12-10T22:30:38.009+05:30 [Info] average scan response {65 ms}
2019-12-10T22:30:38.380+05:30 [Info] CreateIndex 4943923536053865142 default/arr_single using:forestdb exprType:N1QL whereExpr:() secExprs:([(all (`arr_tags`))]) desc:[] isPrimary:false scheme:SINGLE  partitionKeys:([]) with: - elapsed(4.786755606s) err()
2019/12/10 22:30:38 Created the secondary index arr_single. Waiting for it become active
2019-12-10T22:30:38.380+05:30 [Info] metadata provider version changed 1308 -> 1309
2019-12-10T22:30:38.380+05:30 [Info] switched currmeta from 1308 -> 1309 force false 
2019/12/10 22:30:38 Index is now active
2019-12-10T22:30:38.381+05:30 [Info] send prepare create request to watcher 127.0.0.1:9106
2019-12-10T22:30:38.385+05:30 [Info] Indexer 127.0.0.1:9106 accept prepare request. Index (default, arr_leading)
2019-12-10T22:30:38.453+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:38.453+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:38.470+05:30 [Info] switched currmeta from 1304 -> 1305 force true 
2019-12-10T22:30:38.472+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:38.472+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:38.478+05:30 [Info] switched currmeta from 1309 -> 1309 force true 
2019-12-10T22:30:38.532+05:30 [Info] Planner::processCreateToken: There is no create token to process for node 127.0.0.1:9001
2019-12-10T22:30:38.933+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node 127.0.0.1:9001
2019-12-10T22:30:39.011+05:30 [Info] ************ Indexer Layout *************
2019-12-10T22:30:39.011+05:30 [Info] Score: 0
2019-12-10T22:30:39.011+05:30 [Info] variation: 0
2019-12-10T22:30:39.011+05:30 [Info] ElapsedTime: 9ms
2019-12-10T22:30:39.011+05:30 [Info] ConvergenceTime: 2us
2019-12-10T22:30:39.011+05:30 [Info] Iteration: 0
2019-12-10T22:30:39.011+05:30 [Info] Move: 0
2019-12-10T22:30:39.011+05:30 [Info] ----------------------------------------
2019-12-10T22:30:39.012+05:30 [Info] Indexer Memory Mean 1258291200 (1.17188G)
2019-12-10T22:30:39.012+05:30 [Info] Indexer Memory Deviation 0 (0) (0.00%)
2019-12-10T22:30:39.012+05:30 [Info] Indexer Memory Utilization 0.8000
2019-12-10T22:30:39.012+05:30 [Info] Indexer CPU Mean 0.0000
2019-12-10T22:30:39.012+05:30 [Info] Indexer CPU Deviation 0.00 (0.00%)
2019-12-10T22:30:39.012+05:30 [Info] Indexer CPU Utilization 0.0000
2019-12-10T22:30:39.012+05:30 [Info] Indexer IO Mean 0.0000
2019-12-10T22:30:39.012+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2019-12-10T22:30:39.012+05:30 [Info] Indexer Drain Rate Mean 0.0000
2019-12-10T22:30:39.012+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2019-12-10T22:30:39.012+05:30 [Info] Indexer Scan Rate Mean 0.0000
2019-12-10T22:30:39.012+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2019-12-10T22:30:39.012+05:30 [Info] Indexer Data Size Mean 1258980780 (1.17252G)
2019-12-10T22:30:39.012+05:30 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2019-12-10T22:30:39.012+05:30 [Info] Total Index Data (from non-deleted node) 0
2019-12-10T22:30:39.012+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2019-12-10T22:30:39.012+05:30 [Info] No. Index (from non-deleted node) 0
2019-12-10T22:30:39.012+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2019-12-10T22:30:39.012+05:30 [Info] ----------------------------------------
2019-12-10T22:30:39.012+05:30 [Info] Number of indexes: 3
2019-12-10T22:30:39.012+05:30 [Info] Number of indexers: 1
2019-12-10T22:30:39.012+05:30 [Info] Avg Index Size: 419430400 (400M)
2019-12-10T22:30:39.012+05:30 [Info] Max Index Size: 1258291200 (1.17188G)
2019-12-10T22:30:39.012+05:30 [Info] Max Indexer Overhead: 0 (0)
2019-12-10T22:30:39.012+05:30 [Info] Avg Index Cpu: 0.0000
2019-12-10T22:30:39.012+05:30 [Info] Max Index Cpu: 0.0000
2019-12-10T22:30:39.012+05:30 [Info] Num Estimation Run: 1
2019-12-10T22:30:39.012+05:30 [Info] ----------------------------------------
2019-12-10T22:30:39.012+05:30 [Info] Memory Quota 1572864000 (1.46484G)
2019-12-10T22:30:39.012+05:30 [Info] CPU Quota 4
2019-12-10T22:30:39.012+05:30 [Info] Max Cpu Utilization -1
2019-12-10T22:30:39.012+05:30 [Info] Max Memory Utilization -1
2019-12-10T22:30:39.012+05:30 [Info] ----------------------------------------
2019-12-10T22:30:39.012+05:30 [Info] Total iteration: 0
2019-12-10T22:30:39.012+05:30 [Info] RandomSwap time: 0ns
2019-12-10T22:30:39.012+05:30 [Info] RandomSwap call: 0
2019-12-10T22:30:39.012+05:30 [Info] RandomSwap iteration: 0
2019-12-10T22:30:39.012+05:30 [Info] RandomMove time: 0ns
2019-12-10T22:30:39.012+05:30 [Info] RandomMove call: 0 (empty index 0)
2019-12-10T22:30:39.012+05:30 [Info] ExhaustMove time: 0ns
2019-12-10T22:30:39.012+05:30 [Info] ExhaustMove call: 0
2019-12-10T22:30:39.012+05:30 [Info] ExhaustSwap time: 0ns
2019-12-10T22:30:39.012+05:30 [Info] ExhaustSwap call: 0
2019-12-10T22:30:39.012+05:30 [Info] ----------------------------------------
2019-12-10T22:30:39.012+05:30 [Info] 
2019-12-10T22:30:39.012+05:30 [Info] Indexer serverGroup:Group 1, nodeId:127.0.0.1:9001, nodeUUID:280ac5e1026b443deea667c0a12beceb, useLiveData:true
2019-12-10T22:30:39.012+05:30 [Info] Indexer total memory:1258291200 (1.17188G), mem:1258291200 (1.17188G), overhead:0 (0), min:1258423409 (1.172G), data:1258980780 (1.17252G) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:3
2019-12-10T22:30:39.012+05:30 [Info] Indexer isDeleted:false isNew:false exclude: meetConstraint:true
2019-12-10T22:30:39.012+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:30:39.012+05:30 [Info] 		Index name:arr_single 0, bucket:default, defnId:4943923536053865142, instId:14454778974833588294, Partition: 0, new/moved:false
2019-12-10T22:30:39.012+05:30 [Info] 		Index total memory:0 (0), mem:0 (0), overhead:0 (0), min:0 (0), data:28535 (27.8662K) cpu:0.0000 io:0 (0) scan:0 drain:0
2019-12-10T22:30:39.012+05:30 [Info] 		Index resident:0% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:30:39.012+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:30:39.012+05:30 [Info] 		Index name:arridx_friends 0, bucket:default, defnId:11726076518073797138, instId:17505973891200266198, Partition: 0, new/moved:false
2019-12-10T22:30:39.012+05:30 [Info] 		Index total memory:0 (0), mem:0 (0), overhead:0 (0), min:132209 (129.11K), data:661045 (645.552K) cpu:0.0000 io:0 (0) scan:0 drain:0
2019-12-10T22:30:39.012+05:30 [Info] 		Index resident:0% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:30:39.012+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:30:39.012+05:30 [Info] 		Index name:arr_leading 0, bucket:default, defnId:2121931852288146903, instId:1316913403461513271, Partition: 0, new/moved:true
2019-12-10T22:30:39.012+05:30 [Info] 		Index total memory:1258291200 (1.17188G), mem:1258291200 (1.17188G), overhead:0 (0), min:1258291200 (1.17188G), data:1258291200 (1.17188G) cpu:0.0000 io:0 (0) scan:0 drain:0
2019-12-10T22:30:39.012+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:30:39.012+05:30 [Info] ****************************************
2019-12-10T22:30:39.013+05:30 [Info] send commit create request to watcher 127.0.0.1:9106
2019-12-10T22:30:39.161+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:39.161+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:39.164+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:39.164+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:39.165+05:30 [Info] switched currmeta from 1305 -> 1307 force true 
2019-12-10T22:30:39.167+05:30 [Info] switched currmeta from 1309 -> 1311 force true 
2019-12-10T22:30:39.296+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:39.296+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:39.297+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:39.297+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:39.306+05:30 [Info] switched currmeta from 1311 -> 1312 force true 
2019-12-10T22:30:39.307+05:30 [Info] switched currmeta from 1307 -> 1308 force true 
2019-12-10T22:30:39.360+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:39.360+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:39.361+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:39.361+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:39.365+05:30 [Info] switched currmeta from 1308 -> 1309 force true 
2019-12-10T22:30:39.365+05:30 [Info] switched currmeta from 1312 -> 1313 force true 
2019-12-10T22:30:39.525+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:39.526+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:39.525+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:39.526+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:39.529+05:30 [Info] switched currmeta from 1309 -> 1310 force true 
2019-12-10T22:30:39.530+05:30 [Info] switched currmeta from 1313 -> 1314 force true 
2019-12-10T22:30:40.646+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:30:40.651+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] active conns 0, free conns 1
2019-12-10T22:30:40.720+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:40.720+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:40.731+05:30 [Info] switched currmeta from 1310 -> 1310 force true 
2019-12-10T22:30:40.750+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:40.750+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:40.754+05:30 [Info] switched currmeta from 1314 -> 1314 force true 
2019-12-10T22:30:42.121+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:42.121+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:42.136+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:42.136+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:42.139+05:30 [Info] switched currmeta from 1310 -> 1311 force true 
2019-12-10T22:30:42.142+05:30 [Info] switched currmeta from 1314 -> 1315 force true 
2019-12-10T22:30:42.630+05:30 [Info] Rollback time has changed for index inst 14454778974833588294. New rollback time 1575996807857833855
2019-12-10T22:30:42.630+05:30 [Info] Rollback time has changed for index inst 14454778974833588294. New rollback time 1575996807857833855
2019-12-10T22:30:42.942+05:30 [Info] CreateIndex 2121931852288146903 default/arr_leading using:forestdb exprType:N1QL whereExpr:() secExprs:([(all (`arr_tags`)) `arr_name`]) desc:[] isPrimary:false scheme:SINGLE  partitionKeys:([]) with: - elapsed(4.561781598s) err()
2019/12/10 22:30:42 Created the secondary index arr_leading. Waiting for it become active
2019-12-10T22:30:42.943+05:30 [Info] metadata provider version changed 1315 -> 1316
2019-12-10T22:30:42.943+05:30 [Info] switched currmeta from 1315 -> 1316 force false 
2019/12/10 22:30:42 Index is now active
2019-12-10T22:30:42.947+05:30 [Info] send prepare create request to watcher 127.0.0.1:9106
2019-12-10T22:30:42.951+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:42.953+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:42.954+05:30 [Info] Indexer 127.0.0.1:9106 accept prepare request. Index (default, arr_nonleading)
2019-12-10T22:30:42.960+05:30 [Info] switched currmeta from 1311 -> 1312 force true 
2019-12-10T22:30:42.994+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:42.994+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:42.998+05:30 [Info] switched currmeta from 1316 -> 1316 force true 
2019-12-10T22:30:43.059+05:30 [Info] Planner::processCreateToken: There is no create token to process for node 127.0.0.1:9001
2019-12-10T22:30:43.398+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node 127.0.0.1:9001
2019-12-10T22:30:43.483+05:30 [Info] ************ Indexer Layout *************
2019-12-10T22:30:43.483+05:30 [Info] Score: 0
2019-12-10T22:30:43.483+05:30 [Info] variation: 0
2019-12-10T22:30:43.483+05:30 [Info] ElapsedTime: 10ms
2019-12-10T22:30:43.483+05:30 [Info] ConvergenceTime: 3us
2019-12-10T22:30:43.483+05:30 [Info] Iteration: 0
2019-12-10T22:30:43.483+05:30 [Info] Move: 0
2019-12-10T22:30:43.483+05:30 [Info] ----------------------------------------
2019-12-10T22:30:43.483+05:30 [Info] Indexer Memory Mean 1258291200 (1.17188G)
2019-12-10T22:30:43.483+05:30 [Info] Indexer Memory Deviation 0 (0) (0.00%)
2019-12-10T22:30:43.483+05:30 [Info] Indexer Memory Utilization 0.8000
2019-12-10T22:30:43.483+05:30 [Info] Indexer CPU Mean 0.0000
2019-12-10T22:30:43.483+05:30 [Info] Indexer CPU Deviation 0.00 (0.00%)
2019-12-10T22:30:43.483+05:30 [Info] Indexer CPU Utilization 0.0000
2019-12-10T22:30:43.483+05:30 [Info] Indexer IO Mean 0.0000
2019-12-10T22:30:43.483+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2019-12-10T22:30:43.483+05:30 [Info] Indexer Drain Rate Mean 0.0000
2019-12-10T22:30:43.483+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2019-12-10T22:30:43.483+05:30 [Info] Indexer Scan Rate Mean 0.0000
2019-12-10T22:30:43.483+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2019-12-10T22:30:43.483+05:30 [Info] Indexer Data Size Mean 1259009315 (1.17254G)
2019-12-10T22:30:43.483+05:30 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2019-12-10T22:30:43.483+05:30 [Info] Total Index Data (from non-deleted node) 0
2019-12-10T22:30:43.483+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2019-12-10T22:30:43.483+05:30 [Info] No. Index (from non-deleted node) 0
2019-12-10T22:30:43.483+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2019-12-10T22:30:43.483+05:30 [Info] ----------------------------------------
2019-12-10T22:30:43.483+05:30 [Info] Number of indexes: 4
2019-12-10T22:30:43.483+05:30 [Info] Number of indexers: 1
2019-12-10T22:30:43.483+05:30 [Info] Avg Index Size: 314572800 (300M)
2019-12-10T22:30:43.483+05:30 [Info] Max Index Size: 1258291200 (1.17188G)
2019-12-10T22:30:43.483+05:30 [Info] Max Indexer Overhead: 0 (0)
2019-12-10T22:30:43.483+05:30 [Info] Avg Index Cpu: 0.0000
2019-12-10T22:30:43.483+05:30 [Info] Max Index Cpu: 0.0000
2019-12-10T22:30:43.483+05:30 [Info] Num Estimation Run: 1
2019-12-10T22:30:43.483+05:30 [Info] ----------------------------------------
2019-12-10T22:30:43.483+05:30 [Info] Memory Quota 1572864000 (1.46484G)
2019-12-10T22:30:43.483+05:30 [Info] CPU Quota 4
2019-12-10T22:30:43.483+05:30 [Info] Max Cpu Utilization -1
2019-12-10T22:30:43.483+05:30 [Info] Max Memory Utilization -1
2019-12-10T22:30:43.483+05:30 [Info] ----------------------------------------
2019-12-10T22:30:43.483+05:30 [Info] Total iteration: 0
2019-12-10T22:30:43.483+05:30 [Info] RandomSwap time: 0ns
2019-12-10T22:30:43.483+05:30 [Info] RandomSwap call: 0
2019-12-10T22:30:43.483+05:30 [Info] RandomSwap iteration: 0
2019-12-10T22:30:43.483+05:30 [Info] RandomMove time: 0ns
2019-12-10T22:30:43.483+05:30 [Info] RandomMove call: 0 (empty index 0)
2019-12-10T22:30:43.483+05:30 [Info] ExhaustMove time: 0ns
2019-12-10T22:30:43.483+05:30 [Info] ExhaustMove call: 0
2019-12-10T22:30:43.483+05:30 [Info] ExhaustSwap time: 0ns
2019-12-10T22:30:43.483+05:30 [Info] ExhaustSwap call: 0
2019-12-10T22:30:43.483+05:30 [Info] ----------------------------------------
2019-12-10T22:30:43.483+05:30 [Info] 
2019-12-10T22:30:43.483+05:30 [Info] Indexer serverGroup:Group 1, nodeId:127.0.0.1:9001, nodeUUID:280ac5e1026b443deea667c0a12beceb, useLiveData:true
2019-12-10T22:30:43.483+05:30 [Info] Indexer total memory:1258291200 (1.17188G), mem:1258291200 (1.17188G), overhead:0 (0), min:1258423409 (1.172G), data:1259009315 (1.17254G) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:4
2019-12-10T22:30:43.483+05:30 [Info] Indexer isDeleted:false isNew:false exclude: meetConstraint:true
2019-12-10T22:30:43.483+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:30:43.483+05:30 [Info] 		Index name:arridx_friends 0, bucket:default, defnId:11726076518073797138, instId:17505973891200266198, Partition: 0, new/moved:false
2019-12-10T22:30:43.483+05:30 [Info] 		Index total memory:0 (0), mem:0 (0), overhead:0 (0), min:132209 (129.11K), data:661045 (645.552K) cpu:0.0000 io:0 (0) scan:0 drain:0
2019-12-10T22:30:43.483+05:30 [Info] 		Index resident:0% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:30:43.483+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:30:43.483+05:30 [Info] 		Index name:arr_single 0, bucket:default, defnId:4943923536053865142, instId:14454778974833588294, Partition: 0, new/moved:false
2019-12-10T22:30:43.483+05:30 [Info] 		Index total memory:0 (0), mem:0 (0), overhead:0 (0), min:0 (0), data:28535 (27.8662K) cpu:0.0000 io:0 (0) scan:0 drain:0
2019-12-10T22:30:43.483+05:30 [Info] 		Index resident:0% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:30:43.483+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:30:43.483+05:30 [Info] 		Index name:arr_leading 0, bucket:default, defnId:2121931852288146903, instId:17884820151760081554, Partition: 0, new/moved:false
2019-12-10T22:30:43.483+05:30 [Info] 		Index total memory:0 (0), mem:0 (0), overhead:0 (0), min:0 (0), data:28535 (27.8662K) cpu:0.0000 io:0 (0) scan:0 drain:0
2019-12-10T22:30:43.483+05:30 [Info] 		Index resident:0% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:30:43.483+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:30:43.483+05:30 [Info] 		Index name:arr_nonleading 0, bucket:default, defnId:11075272524799529865, instId:9706281951429468614, Partition: 0, new/moved:true
2019-12-10T22:30:43.483+05:30 [Info] 		Index total memory:1258291200 (1.17188G), mem:1258291200 (1.17188G), overhead:0 (0), min:1258291200 (1.17188G), data:1258291200 (1.17188G) cpu:0.0000 io:0 (0) scan:0 drain:0
2019-12-10T22:30:43.483+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:30:43.483+05:30 [Info] ****************************************
2019-12-10T22:30:43.484+05:30 [Info] send commit create request to watcher 127.0.0.1:9106
2019-12-10T22:30:43.613+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:43.613+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:43.618+05:30 [Info] switched currmeta from 1316 -> 1318 force true 
2019-12-10T22:30:43.632+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:43.632+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:43.636+05:30 [Info] switched currmeta from 1312 -> 1314 force true 
2019-12-10T22:30:43.754+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:43.754+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:43.756+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:43.756+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:43.760+05:30 [Info] switched currmeta from 1318 -> 1319 force true 
2019-12-10T22:30:43.761+05:30 [Info] switched currmeta from 1314 -> 1315 force true 
2019-12-10T22:30:43.827+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:43.827+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:43.831+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:43.831+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:43.831+05:30 [Info] switched currmeta from 1315 -> 1316 force true 
2019-12-10T22:30:43.836+05:30 [Info] switched currmeta from 1319 -> 1320 force true 
2019-12-10T22:30:44.016+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:44.016+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:44.022+05:30 [Info] switched currmeta from 1320 -> 1321 force true 
2019-12-10T22:30:44.026+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:44.026+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:44.037+05:30 [Info] switched currmeta from 1316 -> 1317 force true 
2019-12-10T22:30:45.658+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:30:45.719+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:45.719+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:45.728+05:30 [Info] switched currmeta from 1321 -> 1321 force true 
2019-12-10T22:30:45.741+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:45.741+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:45.747+05:30 [Info] switched currmeta from 1317 -> 1317 force true 
2019-12-10T22:30:46.621+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:46.621+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:46.622+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:46.622+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:46.630+05:30 [Info] switched currmeta from 1317 -> 1318 force true 
2019-12-10T22:30:46.630+05:30 [Info] switched currmeta from 1321 -> 1322 force true 
2019-12-10T22:30:47.445+05:30 [Info] CreateIndex 11075272524799529865 default/arr_nonleading using:forestdb exprType:N1QL whereExpr:() secExprs:([`arr_name` (all (`arr_tags`))]) desc:[] isPrimary:false scheme:SINGLE  partitionKeys:([]) with: - elapsed(4.498875929s) err()
2019/12/10 22:30:47 Created the secondary index arr_nonleading. Waiting for it become active
2019-12-10T22:30:47.445+05:30 [Info] metadata provider version changed 1322 -> 1323
2019-12-10T22:30:47.445+05:30 [Info] switched currmeta from 1322 -> 1323 force false 
2019/12/10 22:30:47 Index is now active
2019/12/10 22:30:47 

--------ScanAll for EMPTY array--------
2019-12-10T22:30:47.536+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:47.536+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:47.536+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:47.536+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:47.541+05:30 [Info] switched currmeta from 1318 -> 1319 force true 
2019-12-10T22:30:47.543+05:30 [Info] switched currmeta from 1323 -> 1323 force true 
2019-12-10T22:30:47.634+05:30 [Info] Rollback time has changed for index inst 2804905165679999492. New rollback time 1575996807857833855
2019-12-10T22:30:47.634+05:30 [Info] Rollback time has changed for index inst 17884820151760081554. New rollback time 1575996807857833855
2019-12-10T22:30:47.634+05:30 [Info] Rollback time has changed for index inst 17884820151760081554. New rollback time 1575996807857833855
2019-12-10T22:30:47.634+05:30 [Info] Rollback time has changed for index inst 2804905165679999492. New rollback time 1575996807857833855
2019/12/10 22:30:47 Count of scanResults is 0
2019/12/10 22:30:47 Count of scanResults is 0
2019/12/10 22:30:47 Count of scanResults is 1
2019/12/10 22:30:47 Key: string -4201081275859523631  Value: value.Values ["t8BbK" Missing field or index.] false
2019/12/10 22:30:47 Expected and Actual scan responses are the same
2019/12/10 22:30:47 

--------ScanAll for MISSING array--------
2019/12/10 22:30:47 Count of scanResults is 0
2019/12/10 22:30:47 Count of scanResults is 0
2019/12/10 22:30:47 Count of scanResults is 1
2019/12/10 22:30:47 Key: string -4201081275859523631  Value: value.Values ["0sWOB" Missing field or index.] false
2019/12/10 22:30:47 Expected and Actual scan responses are the same
2019/12/10 22:30:47 

--------ScanAll for NULL array--------
2019/12/10 22:30:48 Count of scanResults is 1
2019/12/10 22:30:48 Key: string -4201081275859523631  Value: value.Values [null] false
2019/12/10 22:30:48 Expected and Actual scan responses are the same
2019/12/10 22:30:48 Count of scanResults is 1
2019/12/10 22:30:48 Key: string -4201081275859523631  Value: value.Values [null "YywvII"] false
2019/12/10 22:30:48 Expected and Actual scan responses are the same
2019/12/10 22:30:48 Count of scanResults is 1
2019/12/10 22:30:48 Key: string -4201081275859523631  Value: value.Values ["YywvII" null] false
2019/12/10 22:30:48 Expected and Actual scan responses are the same
2019/12/10 22:30:48 

--------ScanAll for SCALARVALUE array--------
2019/12/10 22:30:48 Count of scanResults is 1
2019/12/10 22:30:48 Key: string -4201081275859523631  Value: value.Values ["IamScalar"] false
2019/12/10 22:30:48 Expected and Actual scan responses are the same
2019/12/10 22:30:48 Count of scanResults is 1
2019/12/10 22:30:48 Key: string -4201081275859523631  Value: value.Values ["IamScalar" "pWoN8Y"] false
2019/12/10 22:30:48 Expected and Actual scan responses are the same
2019/12/10 22:30:48 Count of scanResults is 1
2019/12/10 22:30:48 Key: string -4201081275859523631  Value: value.Values ["pWoN8Y" "IamScalar"] false
2019/12/10 22:30:48 Expected and Actual scan responses are the same
2019/12/10 22:30:48 

--------ScanAll for SCALAROBJECT array--------
2019/12/10 22:30:48 Count of scanResults is 1
2019/12/10 22:30:48 Key: string -4201081275859523631  Value: value.Values [{"1":"abc","2":"def"}] false
2019/12/10 22:30:48 Expected and Actual scan responses are the same
2019/12/10 22:30:48 Count of scanResults is 1
2019/12/10 22:30:48 Key: string -4201081275859523631  Value: value.Values [{"1":"abc","2":"def"} "5V3pRs"] false
2019/12/10 22:30:48 Expected and Actual scan responses are the same
2019/12/10 22:30:48 Count of scanResults is 1
2019/12/10 22:30:48 Key: string -4201081275859523631  Value: value.Values ["5V3pRs" {"1":"abc","2":"def"}] false
2019/12/10 22:30:48 Expected and Actual scan responses are the same
--- PASS: TestArrayIndexCornerCases (14.88s)
=== RUN   TestArraySizeIncreaseDecrease1
2019/12/10 22:30:48 In TestArraySizeIncreaseDecrease1()
2019/12/10 22:30:48 In DropAllSecondaryIndexes()
2019/12/10 22:30:48 Index found:  arridx_friends
2019-12-10T22:30:48.595+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:48.595+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:48.596+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:48.596+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:48.600+05:30 [Info] switched currmeta from 1319 -> 1320 force true 
2019-12-10T22:30:48.600+05:30 [Info] switched currmeta from 1323 -> 1324 force true 
2019-12-10T22:30:48.653+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:48.654+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:48.656+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:48.656+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:48.658+05:30 [Info] switched currmeta from 1324 -> 1324 force true 
2019-12-10T22:30:48.660+05:30 [Info] switched currmeta from 1320 -> 1320 force true 
2019-12-10T22:30:48.669+05:30 [Info] metadata provider version changed 1324 -> 1325
2019-12-10T22:30:48.669+05:30 [Info] switched currmeta from 1324 -> 1325 force false 
2019-12-10T22:30:48.669+05:30 [Info] DropIndex 11726076518073797138 - elapsed(197.400637ms), err()
2019/12/10 22:30:48 Dropped index arridx_friends
2019/12/10 22:30:48 Index found:  arr_single
2019-12-10T22:30:48.771+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:48.771+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:48.772+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:48.772+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:48.775+05:30 [Info] switched currmeta from 1325 -> 1326 force true 
2019-12-10T22:30:48.777+05:30 [Info] switched currmeta from 1320 -> 1322 force true 
2019-12-10T22:30:48.829+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:48.829+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:48.836+05:30 [Info] switched currmeta from 1322 -> 1322 force true 
2019-12-10T22:30:48.844+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:48.844+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:48.847+05:30 [Info] switched currmeta from 1326 -> 1326 force true 
2019-12-10T22:30:48.886+05:30 [Info] metadata provider version changed 1326 -> 1327
2019-12-10T22:30:48.886+05:30 [Info] switched currmeta from 1326 -> 1327 force false 
2019-12-10T22:30:48.886+05:30 [Info] DropIndex 4943923536053865142 - elapsed(217.087605ms), err()
2019/12/10 22:30:48 Dropped index arr_single
2019/12/10 22:30:48 Index found:  arr_nonleading
2019-12-10T22:30:48.902+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:48.902+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:48.907+05:30 [Info] switched currmeta from 1322 -> 1323 force true 
2019-12-10T22:30:48.920+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:48.920+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:48.929+05:30 [Info] switched currmeta from 1327 -> 1327 force true 
2019-12-10T22:30:48.989+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:48.989+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:48.999+05:30 [Info] switched currmeta from 1323 -> 1324 force true 
2019-12-10T22:30:49.004+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:49.004+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:49.011+05:30 [Info] switched currmeta from 1327 -> 1328 force true 
2019-12-10T22:30:49.042+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:49.042+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:49.053+05:30 [Info] switched currmeta from 1324 -> 1324 force true 
2019-12-10T22:30:49.054+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:49.055+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:49.057+05:30 [Info] switched currmeta from 1328 -> 1328 force true 
2019-12-10T22:30:49.131+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:49.131+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:49.133+05:30 [Info] switched currmeta from 1324 -> 1324 force true 
2019-12-10T22:30:49.134+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:49.134+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:49.137+05:30 [Info] switched currmeta from 1328 -> 1328 force true 
2019-12-10T22:30:49.139+05:30 [Info] metadata provider version changed 1328 -> 1329
2019-12-10T22:30:49.139+05:30 [Info] switched currmeta from 1328 -> 1329 force false 
2019-12-10T22:30:49.139+05:30 [Info] DropIndex 11075272524799529865 - elapsed(253.088391ms), err()
2019/12/10 22:30:49 Dropped index arr_nonleading
2019/12/10 22:30:49 Index found:  arr_leading
2019-12-10T22:30:49.214+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:49.214+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:49.218+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:49.218+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:49.219+05:30 [Info] switched currmeta from 1324 -> 1325 force true 
2019-12-10T22:30:49.221+05:30 [Info] switched currmeta from 1329 -> 1329 force true 
2019-12-10T22:30:49.280+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:49.281+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:49.291+05:30 [Info] switched currmeta from 1329 -> 1330 force true 
2019-12-10T22:30:49.302+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:49.302+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:49.306+05:30 [Info] switched currmeta from 1325 -> 1326 force true 
2019-12-10T22:30:49.357+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:49.357+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:49.362+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:49.362+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:49.363+05:30 [Info] switched currmeta from 1330 -> 1330 force true 
2019-12-10T22:30:49.365+05:30 [Info] switched currmeta from 1326 -> 1326 force true 
2019-12-10T22:30:49.390+05:30 [Info] metadata provider version changed 1330 -> 1331
2019-12-10T22:30:49.390+05:30 [Info] switched currmeta from 1330 -> 1331 force false 
2019-12-10T22:30:49.390+05:30 [Info] DropIndex 2121931852288146903 - elapsed(251.009758ms), err()
2019/12/10 22:30:49 Dropped index arr_leading
2019-12-10T22:30:49.537+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:49.537+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:49.546+05:30 [Info] switched currmeta from 1326 -> 1327 force true 
2019-12-10T22:30:49.547+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:49.547+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:49.560+05:30 [Info] switched currmeta from 1331 -> 1331 force true 
2019-12-10T22:30:50.405+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:30:50.466+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:50.466+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:50.471+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:50.471+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:50.472+05:30 [Info] switched currmeta from 1331 -> 1331 force true 
2019-12-10T22:30:50.475+05:30 [Info] switched currmeta from 1327 -> 1327 force true 
2019-12-10T22:30:50.642+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:30:50.680+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:50.680+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:50.682+05:30 [Info] switched currmeta from 1327 -> 1327 force true 
2019-12-10T22:30:50.684+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:50.684+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:50.687+05:30 [Info] switched currmeta from 1331 -> 1331 force true 
2019-12-10T22:30:52.553+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:30:52.602+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:52.602+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:52.605+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:52.605+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:52.614+05:30 [Info] switched currmeta from 1331 -> 1331 force true 
2019-12-10T22:30:52.615+05:30 [Info] switched currmeta from 1327 -> 1327 force true 
2019-12-10T22:30:55.641+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:30:55.673+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:55.673+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:55.675+05:30 [Info] switched currmeta from 1327 -> 1327 force true 
2019-12-10T22:30:55.676+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:30:55.676+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:30:55.680+05:30 [Info] switched currmeta from 1331 -> 1331 force true 
2019-12-10T22:30:55.866+05:30 [Info] connected with 1 indexers
2019-12-10T22:30:55.866+05:30 [Info] client load stats {}
2019-12-10T22:30:55.866+05:30 [Info] client hit stats {}
2019-12-10T22:30:55.866+05:30 [Info] client pending item stats {}
2019-12-10T22:30:55.866+05:30 [Info] client stats current {}
2019-12-10T22:30:55.874+05:30 [Info] num concurrent scans {0}
2019-12-10T22:30:55.874+05:30 [Info] average scan response {25 ms}
2019-12-10T22:30:58.462+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] active conns 0, free conns 1
2019-12-10T22:31:00.398+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:31:00.499+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:31:00.499+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:31:00.504+05:30 [Info] switched currmeta from 1331 -> 1331 force true 
2019-12-10T22:31:00.504+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:31:00.504+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:31:00.513+05:30 [Info] switched currmeta from 1327 -> 1327 force true 
2019-12-10T22:31:02.549+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:31:02.594+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:31:02.594+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:31:02.596+05:30 [Info] switched currmeta from 1331 -> 1331 force true 
2019-12-10T22:31:02.611+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:31:02.612+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:31:02.616+05:30 [Info] switched currmeta from 1327 -> 1327 force true 
2019-12-10T22:31:38.006+05:30 [Info] connected with 1 indexers
2019-12-10T22:31:38.006+05:30 [Info] client load stats {}
2019-12-10T22:31:38.006+05:30 [Info] client hit stats {}
2019-12-10T22:31:38.006+05:30 [Info] client pending item stats {}
2019-12-10T22:31:38.006+05:30 [Info] client stats current {}
2019-12-10T22:31:38.009+05:30 [Info] num concurrent scans {0}
2019-12-10T22:31:38.009+05:30 [Info] average scan response {65 ms}
2019-12-10T22:31:40.671+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] active conns 0, free conns 1
2019-12-10T22:31:55.866+05:30 [Info] connected with 1 indexers
2019-12-10T22:31:55.866+05:30 [Info] client load stats {}
2019-12-10T22:31:55.866+05:30 [Info] client hit stats {}
2019-12-10T22:31:55.866+05:30 [Info] client pending item stats {}
2019-12-10T22:31:55.866+05:30 [Info] client stats current {}
2019-12-10T22:31:55.874+05:30 [Info] num concurrent scans {0}
2019-12-10T22:31:55.874+05:30 [Info] average scan response {25 ms}
2019/12/10 22:31:57 Flushed the bucket default
2019-12-10T22:31:57.901+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9106 connected=true
2019/12/10 22:31:57 DBG: ChangeIndexerSettings: nodes = [0xc03c5275c0]
2019/12/10 22:31:57 DBG: ChangeIndexerSettings: adminurl = 127.0.0.1:9106 host 127.0.0.1 sport 9106
2019/12/10 22:31:57 Changing config key indexer.settings.allow_large_keys to value false
2019-12-10T22:31:57.921+05:30 [Info] New settings received: 
{"indexer.api.enableTestServer":true,"indexer.settings.allow_large_keys":false,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfFname":"","indexer.settings.cpuProfile":false,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfFname":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":250,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.recovery.max_rollbacks":5,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":0,"indexer.settings.send_buffer_size":1024,"indexer.settings.sliceBufSize":50000,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"forestdb","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.wal_size":4096,"projector.settings.log_level":"info","queryport.client.settings.backfillLimit":0,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100}
2019-12-10T22:31:58.480+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] active conns 0, free conns 1
2019-12-10T22:31:58.913+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9106 connected=true
2019/12/10 22:31:58 DBG: ChangeIndexerSettings: nodes = [0xc030259300]
2019/12/10 22:31:58 DBG: ChangeIndexerSettings: adminurl = 127.0.0.1:9106 host 127.0.0.1 sport 9106
2019/12/10 22:31:58 Changing config key indexer.settings.max_seckey_size to value 100
2019-12-10T22:31:58.924+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9106 connected=true
2019/12/10 22:31:58 DBG: ChangeIndexerSettings: nodes = [0xc02fa1e8c0]
2019/12/10 22:31:58 DBG: ChangeIndexerSettings: adminurl = 127.0.0.1:9106 host 127.0.0.1 sport 9106
2019/12/10 22:31:58 Changing config key indexer.settings.max_array_seckey_size to value 2000
2019-12-10T22:31:58.938+05:30 [Info] New settings received: 
{"indexer.api.enableTestServer":true,"indexer.settings.allow_large_keys":false,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfFname":"","indexer.settings.cpuProfile":false,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":100,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfFname":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":250,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.recovery.max_rollbacks":5,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":0,"indexer.settings.send_buffer_size":1024,"indexer.settings.sliceBufSize":50000,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"forestdb","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.wal_size":4096,"projector.settings.log_level":"info","queryport.client.settings.backfillLimit":0,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100}
2019-12-10T22:31:58.999+05:30 [Info] New settings received: 
{"indexer.api.enableTestServer":true,"indexer.settings.allow_large_keys":false,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfFname":"","indexer.settings.cpuProfile":false,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":2000,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":100,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfFname":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":250,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.recovery.max_rollbacks":5,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":0,"indexer.settings.send_buffer_size":1024,"indexer.settings.sliceBufSize":50000,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"forestdb","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.wal_size":4096,"projector.settings.log_level":"info","queryport.client.settings.backfillLimit":0,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100}
2019/12/10 22:31:59 Start of createArrayDocs()
2019/12/10 22:32:17 End of createArrayDocs()
2019/12/10 22:32:17 Start of createArrayDocs()
2019/12/10 22:32:17 End of createArrayDocs()
2019-12-10T22:32:17.594+05:30 [Info] send prepare create request to watcher 127.0.0.1:9106
2019-12-10T22:32:17.596+05:30 [Info] Indexer 127.0.0.1:9106 accept prepare request. Index (default, arr1)
2019-12-10T22:32:17.665+05:30 [Info] Planner::processCreateToken: There is no create token to process for node 127.0.0.1:9001
2019-12-10T22:32:18.020+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node 127.0.0.1:9001
2019-12-10T22:32:18.086+05:30 [Info] ************ Indexer Layout *************
2019-12-10T22:32:18.086+05:30 [Info] Score: 0
2019-12-10T22:32:18.086+05:30 [Info] variation: 0
2019-12-10T22:32:18.086+05:30 [Info] ElapsedTime: 5ms
2019-12-10T22:32:18.086+05:30 [Info] ConvergenceTime: 2us
2019-12-10T22:32:18.086+05:30 [Info] Iteration: 0
2019-12-10T22:32:18.086+05:30 [Info] Move: 0
2019-12-10T22:32:18.086+05:30 [Info] ----------------------------------------
2019-12-10T22:32:18.086+05:30 [Info] Indexer Memory Mean 1258291200 (1.17188G)
2019-12-10T22:32:18.086+05:30 [Info] Indexer Memory Deviation 0 (0) (0.00%)
2019-12-10T22:32:18.086+05:30 [Info] Indexer Memory Utilization 0.8000
2019-12-10T22:32:18.086+05:30 [Info] Indexer CPU Mean 0.0000
2019-12-10T22:32:18.086+05:30 [Info] Indexer CPU Deviation 0.00 (0.00%)
2019-12-10T22:32:18.086+05:30 [Info] Indexer CPU Utilization 0.0000
2019-12-10T22:32:18.086+05:30 [Info] Indexer IO Mean 0.0000
2019-12-10T22:32:18.086+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2019-12-10T22:32:18.086+05:30 [Info] Indexer Drain Rate Mean 0.0000
2019-12-10T22:32:18.086+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2019-12-10T22:32:18.086+05:30 [Info] Indexer Scan Rate Mean 0.0000
2019-12-10T22:32:18.086+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2019-12-10T22:32:18.086+05:30 [Info] Indexer Data Size Mean 1258291200 (1.17188G)
2019-12-10T22:32:18.086+05:30 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2019-12-10T22:32:18.086+05:30 [Info] Total Index Data (from non-deleted node) 0
2019-12-10T22:32:18.086+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2019-12-10T22:32:18.086+05:30 [Info] No. Index (from non-deleted node) 0
2019-12-10T22:32:18.086+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2019-12-10T22:32:18.086+05:30 [Info] ----------------------------------------
2019-12-10T22:32:18.086+05:30 [Info] Number of indexes: 1
2019-12-10T22:32:18.086+05:30 [Info] Number of indexers: 1
2019-12-10T22:32:18.086+05:30 [Info] Avg Index Size: 1258291200 (1.17188G)
2019-12-10T22:32:18.086+05:30 [Info] Max Index Size: 1258291200 (1.17188G)
2019-12-10T22:32:18.086+05:30 [Info] Max Indexer Overhead: 0 (0)
2019-12-10T22:32:18.086+05:30 [Info] Avg Index Cpu: 0.0000
2019-12-10T22:32:18.086+05:30 [Info] Max Index Cpu: 0.0000
2019-12-10T22:32:18.086+05:30 [Info] Num Estimation Run: 1
2019-12-10T22:32:18.086+05:30 [Info] ----------------------------------------
2019-12-10T22:32:18.086+05:30 [Info] Memory Quota 1572864000 (1.46484G)
2019-12-10T22:32:18.086+05:30 [Info] CPU Quota 4
2019-12-10T22:32:18.086+05:30 [Info] Max Cpu Utilization -1
2019-12-10T22:32:18.086+05:30 [Info] Max Memory Utilization -1
2019-12-10T22:32:18.086+05:30 [Info] ----------------------------------------
2019-12-10T22:32:18.086+05:30 [Info] Total iteration: 0
2019-12-10T22:32:18.086+05:30 [Info] RandomSwap time: 0ns
2019-12-10T22:32:18.086+05:30 [Info] RandomSwap call: 0
2019-12-10T22:32:18.086+05:30 [Info] RandomSwap iteration: 0
2019-12-10T22:32:18.086+05:30 [Info] RandomMove time: 0ns
2019-12-10T22:32:18.086+05:30 [Info] RandomMove call: 0 (empty index 0)
2019-12-10T22:32:18.086+05:30 [Info] ExhaustMove time: 0ns
2019-12-10T22:32:18.087+05:30 [Info] ExhaustMove call: 0
2019-12-10T22:32:18.087+05:30 [Info] ExhaustSwap time: 0ns
2019-12-10T22:32:18.087+05:30 [Info] ExhaustSwap call: 0
2019-12-10T22:32:18.087+05:30 [Info] ----------------------------------------
2019-12-10T22:32:18.087+05:30 [Info] 
2019-12-10T22:32:18.087+05:30 [Info] Indexer serverGroup:Group 1, nodeId:127.0.0.1:9001, nodeUUID:280ac5e1026b443deea667c0a12beceb, useLiveData:true
2019-12-10T22:32:18.087+05:30 [Info] Indexer total memory:1258291200 (1.17188G), mem:1258291200 (1.17188G), overhead:0 (0), min:1258291200 (1.17188G), data:1258291200 (1.17188G) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:1
2019-12-10T22:32:18.087+05:30 [Info] Indexer isDeleted:false isNew:false exclude: meetConstraint:true
2019-12-10T22:32:18.087+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:32:18.087+05:30 [Info] 		Index name:arr1 0, bucket:default, defnId:1619686062513106516, instId:10109936538247232294, Partition: 0, new/moved:true
2019-12-10T22:32:18.087+05:30 [Info] 		Index total memory:1258291200 (1.17188G), mem:1258291200 (1.17188G), overhead:0 (0), min:1258291200 (1.17188G), data:1258291200 (1.17188G) cpu:0.0000 io:0 (0) scan:0 drain:0
2019-12-10T22:32:18.087+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:32:18.087+05:30 [Info] ****************************************
2019-12-10T22:32:18.087+05:30 [Info] send commit create request to watcher 127.0.0.1:9106
2019-12-10T22:32:18.233+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:32:18.233+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:32:18.236+05:30 [Info] switched currmeta from 1331 -> 1333 force true 
2019-12-10T22:32:18.241+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:32:18.241+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:32:18.247+05:30 [Info] switched currmeta from 1327 -> 1329 force true 
2019-12-10T22:32:18.397+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:32:18.397+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:32:18.400+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:32:18.401+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:32:18.404+05:30 [Info] switched currmeta from 1333 -> 1335 force true 
2019-12-10T22:32:18.405+05:30 [Info] switched currmeta from 1329 -> 1331 force true 
2019-12-10T22:32:18.459+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:32:18.459+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:32:18.463+05:30 [Info] switched currmeta from 1335 -> 1335 force true 
2019-12-10T22:32:18.464+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:32:18.464+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:32:18.468+05:30 [Info] switched currmeta from 1331 -> 1331 force true 
2019-12-10T22:32:18.602+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:32:18.602+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:32:18.604+05:30 [Info] switched currmeta from 1331 -> 1332 force true 
2019-12-10T22:32:18.608+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:32:18.608+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:32:18.609+05:30 [Info] switched currmeta from 1335 -> 1336 force true 
2019-12-10T22:32:22.634+05:30 [Info] Rollback time has changed for index inst 8082202379049928859. New rollback time 1575996807857833855
2019-12-10T22:32:22.634+05:30 [Info] Rollback time has changed for index inst 8082202379049928859. New rollback time 1575996807857833855
2019-12-10T22:32:22.725+05:30 [Info] CreateIndex 1619686062513106516 default/arr1 using:forestdb exprType:N1QL whereExpr:() secExprs:([(all (`friends`))]) desc:[] isPrimary:false scheme:SINGLE  partitionKeys:([]) with: - elapsed(5.13154105s) err()
2019/12/10 22:32:22 Created the secondary index arr1. Waiting for it become active
2019-12-10T22:32:22.725+05:30 [Info] metadata provider version changed 1336 -> 1337
2019-12-10T22:32:22.725+05:30 [Info] switched currmeta from 1336 -> 1337 force false 
2019/12/10 22:32:22 Index is now active
2019-12-10T22:32:22.726+05:30 [Info] send prepare create request to watcher 127.0.0.1:9106
2019-12-10T22:32:22.727+05:30 [Info] Indexer 127.0.0.1:9106 accept prepare request. Index (default, arr2)
2019-12-10T22:32:22.779+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:32:22.779+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:32:22.782+05:30 [Info] switched currmeta from 1332 -> 1333 force true 
2019-12-10T22:32:22.792+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:32:22.792+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:32:22.796+05:30 [Info] switched currmeta from 1337 -> 1337 force true 
2019-12-10T22:32:22.815+05:30 [Info] Planner::processCreateToken: There is no create token to process for node 127.0.0.1:9001
2019-12-10T22:32:23.130+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node 127.0.0.1:9001
2019-12-10T22:32:23.203+05:30 [Info] ************ Indexer Layout *************
2019-12-10T22:32:23.203+05:30 [Info] Score: 0
2019-12-10T22:32:23.203+05:30 [Info] variation: 0
2019-12-10T22:32:23.203+05:30 [Info] ElapsedTime: 7ms
2019-12-10T22:32:23.203+05:30 [Info] ConvergenceTime: 2us
2019-12-10T22:32:23.203+05:30 [Info] Iteration: 0
2019-12-10T22:32:23.203+05:30 [Info] Move: 0
2019-12-10T22:32:23.203+05:30 [Info] ----------------------------------------
2019-12-10T22:32:23.203+05:30 [Info] Indexer Memory Mean 1258291200 (1.17188G)
2019-12-10T22:32:23.203+05:30 [Info] Indexer Memory Deviation 0 (0) (0.00%)
2019-12-10T22:32:23.203+05:30 [Info] Indexer Memory Utilization 0.8000
2019-12-10T22:32:23.203+05:30 [Info] Indexer CPU Mean 0.0000
2019-12-10T22:32:23.203+05:30 [Info] Indexer CPU Deviation 0.00 (0.00%)
2019-12-10T22:32:23.203+05:30 [Info] Indexer CPU Utilization 0.0000
2019-12-10T22:32:23.203+05:30 [Info] Indexer IO Mean 0.0000
2019-12-10T22:32:23.203+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2019-12-10T22:32:23.203+05:30 [Info] Indexer Drain Rate Mean 0.0000
2019-12-10T22:32:23.203+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2019-12-10T22:32:23.203+05:30 [Info] Indexer Scan Rate Mean 0.0000
2019-12-10T22:32:23.203+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2019-12-10T22:32:23.203+05:30 [Info] Indexer Data Size Mean 1258332783 (1.17191G)
2019-12-10T22:32:23.203+05:30 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2019-12-10T22:32:23.203+05:30 [Info] Total Index Data (from non-deleted node) 0
2019-12-10T22:32:23.203+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2019-12-10T22:32:23.203+05:30 [Info] No. Index (from non-deleted node) 0
2019-12-10T22:32:23.203+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2019-12-10T22:32:23.203+05:30 [Info] ----------------------------------------
2019-12-10T22:32:23.203+05:30 [Info] Number of indexes: 2
2019-12-10T22:32:23.203+05:30 [Info] Number of indexers: 1
2019-12-10T22:32:23.203+05:30 [Info] Avg Index Size: 629145600 (600M)
2019-12-10T22:32:23.203+05:30 [Info] Max Index Size: 1258291200 (1.17188G)
2019-12-10T22:32:23.203+05:30 [Info] Max Indexer Overhead: 0 (0)
2019-12-10T22:32:23.203+05:30 [Info] Avg Index Cpu: 0.0000
2019-12-10T22:32:23.203+05:30 [Info] Max Index Cpu: 0.0000
2019-12-10T22:32:23.203+05:30 [Info] Num Estimation Run: 1
2019-12-10T22:32:23.203+05:30 [Info] ----------------------------------------
2019-12-10T22:32:23.203+05:30 [Info] Memory Quota 1572864000 (1.46484G)
2019-12-10T22:32:23.203+05:30 [Info] CPU Quota 4
2019-12-10T22:32:23.203+05:30 [Info] Max Cpu Utilization -1
2019-12-10T22:32:23.203+05:30 [Info] Max Memory Utilization -1
2019-12-10T22:32:23.203+05:30 [Info] ----------------------------------------
2019-12-10T22:32:23.203+05:30 [Info] Total iteration: 0
2019-12-10T22:32:23.203+05:30 [Info] RandomSwap time: 0ns
2019-12-10T22:32:23.203+05:30 [Info] RandomSwap call: 0
2019-12-10T22:32:23.203+05:30 [Info] RandomSwap iteration: 0
2019-12-10T22:32:23.203+05:30 [Info] RandomMove time: 0ns
2019-12-10T22:32:23.203+05:30 [Info] RandomMove call: 0 (empty index 0)
2019-12-10T22:32:23.203+05:30 [Info] ExhaustMove time: 0ns
2019-12-10T22:32:23.203+05:30 [Info] ExhaustMove call: 0
2019-12-10T22:32:23.203+05:30 [Info] ExhaustSwap time: 0ns
2019-12-10T22:32:23.203+05:30 [Info] ExhaustSwap call: 0
2019-12-10T22:32:23.203+05:30 [Info] ----------------------------------------
2019-12-10T22:32:23.203+05:30 [Info] 
2019-12-10T22:32:23.203+05:30 [Info] Indexer serverGroup:Group 1, nodeId:127.0.0.1:9001, nodeUUID:280ac5e1026b443deea667c0a12beceb, useLiveData:true
2019-12-10T22:32:23.203+05:30 [Info] Indexer total memory:1258291200 (1.17188G), mem:1258291200 (1.17188G), overhead:0 (0), min:1258299516 (1.17188G), data:1258332783 (1.17191G) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:2
2019-12-10T22:32:23.203+05:30 [Info] Indexer isDeleted:false isNew:false exclude: meetConstraint:true
2019-12-10T22:32:23.203+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:32:23.203+05:30 [Info] 		Index name:arr1 0, bucket:default, defnId:1619686062513106516, instId:8082202379049928859, Partition: 0, new/moved:false
2019-12-10T22:32:23.203+05:30 [Info] 		Index total memory:0 (0), mem:0 (0), overhead:0 (0), min:8316 (8.12109K), data:41583 (40.6084K) cpu:0.0000 io:0 (0) scan:0 drain:0
2019-12-10T22:32:23.203+05:30 [Info] 		Index resident:0% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:32:23.203+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:32:23.204+05:30 [Info] 		Index name:arr2 0, bucket:default, defnId:4823915220527584383, instId:15990505021826219459, Partition: 0, new/moved:true
2019-12-10T22:32:23.204+05:30 [Info] 		Index total memory:1258291200 (1.17188G), mem:1258291200 (1.17188G), overhead:0 (0), min:1258291200 (1.17188G), data:1258291200 (1.17188G) cpu:0.0000 io:0 (0) scan:0 drain:0
2019-12-10T22:32:23.204+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:32:23.204+05:30 [Info] ****************************************
2019-12-10T22:32:23.204+05:30 [Info] send commit create request to watcher 127.0.0.1:9106
2019-12-10T22:32:23.336+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:32:23.336+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:32:23.341+05:30 [Info] switched currmeta from 1337 -> 1339 force true 
2019-12-10T22:32:23.346+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:32:23.346+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:32:23.350+05:30 [Info] switched currmeta from 1333 -> 1335 force true 
2019-12-10T22:32:23.472+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:32:23.472+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:32:23.480+05:30 [Info] switched currmeta from 1339 -> 1340 force true 
2019-12-10T22:32:23.485+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:32:23.485+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:32:23.489+05:30 [Info] switched currmeta from 1335 -> 1336 force true 
2019-12-10T22:32:23.535+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:32:23.535+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:32:23.536+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:32:23.536+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:32:23.541+05:30 [Info] switched currmeta from 1336 -> 1337 force true 
2019-12-10T22:32:23.541+05:30 [Info] switched currmeta from 1340 -> 1341 force true 
2019-12-10T22:32:23.704+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:32:23.704+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:32:23.705+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:32:23.705+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:32:23.708+05:30 [Info] switched currmeta from 1341 -> 1342 force true 
2019-12-10T22:32:23.728+05:30 [Info] switched currmeta from 1337 -> 1338 force true 
2019-12-10T22:32:25.669+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:32:25.719+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:32:25.719+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:32:25.726+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:32:25.726+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:32:25.731+05:30 [Info] switched currmeta from 1338 -> 1338 force true 
2019-12-10T22:32:25.740+05:30 [Info] switched currmeta from 1342 -> 1342 force true 
2019-12-10T22:32:26.083+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:32:26.083+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:32:26.090+05:30 [Info] switched currmeta from 1342 -> 1343 force true 
2019-12-10T22:32:26.093+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:32:26.093+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:32:26.095+05:30 [Info] switched currmeta from 1338 -> 1339 force true 
2019-12-10T22:32:26.968+05:30 [Info] CreateIndex 4823915220527584383 default/arr2 using:forestdb exprType:N1QL whereExpr:() secExprs:([(all (`name`))]) desc:[] isPrimary:false scheme:SINGLE  partitionKeys:([]) with: - elapsed(4.242782559s) err()
2019/12/10 22:32:26 Created the secondary index arr2. Waiting for it become active
2019-12-10T22:32:26.968+05:30 [Info] metadata provider version changed 1343 -> 1344
2019-12-10T22:32:26.968+05:30 [Info] switched currmeta from 1343 -> 1344 force false 
2019/12/10 22:32:26 Index is now active
2019-12-10T22:32:26.969+05:30 [Info] send prepare create request to watcher 127.0.0.1:9106
2019-12-10T22:32:26.971+05:30 [Info] Indexer 127.0.0.1:9106 accept prepare request. Index (default, idx3)
2019-12-10T22:32:27.032+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:32:27.032+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:32:27.035+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:32:27.035+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:32:27.039+05:30 [Info] switched currmeta from 1339 -> 1340 force true 
2019-12-10T22:32:27.039+05:30 [Info] switched currmeta from 1344 -> 1344 force true 
2019-12-10T22:32:27.079+05:30 [Info] Planner::processCreateToken: There is no create token to process for node 127.0.0.1:9001
2019-12-10T22:32:27.406+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node 127.0.0.1:9001
2019-12-10T22:32:27.490+05:30 [Info] ************ Indexer Layout *************
2019-12-10T22:32:27.490+05:30 [Info] Score: 0
2019-12-10T22:32:27.490+05:30 [Info] variation: 0
2019-12-10T22:32:27.490+05:30 [Info] ElapsedTime: 11ms
2019-12-10T22:32:27.490+05:30 [Info] ConvergenceTime: 2us
2019-12-10T22:32:27.490+05:30 [Info] Iteration: 0
2019-12-10T22:32:27.490+05:30 [Info] Move: 0
2019-12-10T22:32:27.490+05:30 [Info] ----------------------------------------
2019-12-10T22:32:27.490+05:30 [Info] Indexer Memory Mean 1258291200 (1.17188G)
2019-12-10T22:32:27.490+05:30 [Info] Indexer Memory Deviation 0 (0) (0.00%)
2019-12-10T22:32:27.490+05:30 [Info] Indexer Memory Utilization 0.8000
2019-12-10T22:32:27.490+05:30 [Info] Indexer CPU Mean 0.0000
2019-12-10T22:32:27.490+05:30 [Info] Indexer CPU Deviation 0.00 (0.00%)
2019-12-10T22:32:27.490+05:30 [Info] Indexer CPU Utilization 0.0000
2019-12-10T22:32:27.490+05:30 [Info] Indexer IO Mean 0.0000
2019-12-10T22:32:27.490+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2019-12-10T22:32:27.490+05:30 [Info] Indexer Drain Rate Mean 0.0000
2019-12-10T22:32:27.490+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2019-12-10T22:32:27.490+05:30 [Info] Indexer Scan Rate Mean 0.0000
2019-12-10T22:32:27.490+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2019-12-10T22:32:27.490+05:30 [Info] Indexer Data Size Mean 1258367476 (1.17195G)
2019-12-10T22:32:27.490+05:30 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2019-12-10T22:32:27.490+05:30 [Info] Total Index Data (from non-deleted node) 0
2019-12-10T22:32:27.490+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2019-12-10T22:32:27.490+05:30 [Info] No. Index (from non-deleted node) 0
2019-12-10T22:32:27.490+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2019-12-10T22:32:27.490+05:30 [Info] ----------------------------------------
2019-12-10T22:32:27.490+05:30 [Info] Number of indexes: 3
2019-12-10T22:32:27.490+05:30 [Info] Number of indexers: 1
2019-12-10T22:32:27.490+05:30 [Info] Avg Index Size: 419430400 (400M)
2019-12-10T22:32:27.490+05:30 [Info] Max Index Size: 1258291200 (1.17188G)
2019-12-10T22:32:27.490+05:30 [Info] Max Indexer Overhead: 0 (0)
2019-12-10T22:32:27.490+05:30 [Info] Avg Index Cpu: 0.0000
2019-12-10T22:32:27.490+05:30 [Info] Max Index Cpu: 0.0000
2019-12-10T22:32:27.490+05:30 [Info] Num Estimation Run: 1
2019-12-10T22:32:27.490+05:30 [Info] ----------------------------------------
2019-12-10T22:32:27.490+05:30 [Info] Memory Quota 1572864000 (1.46484G)
2019-12-10T22:32:27.490+05:30 [Info] CPU Quota 4
2019-12-10T22:32:27.490+05:30 [Info] Max Cpu Utilization -1
2019-12-10T22:32:27.490+05:30 [Info] Max Memory Utilization -1
2019-12-10T22:32:27.491+05:30 [Info] ----------------------------------------
2019-12-10T22:32:27.491+05:30 [Info] Total iteration: 0
2019-12-10T22:32:27.491+05:30 [Info] RandomSwap time: 0ns
2019-12-10T22:32:27.491+05:30 [Info] RandomSwap call: 0
2019-12-10T22:32:27.491+05:30 [Info] RandomSwap iteration: 0
2019-12-10T22:32:27.491+05:30 [Info] RandomMove time: 0ns
2019-12-10T22:32:27.491+05:30 [Info] RandomMove call: 0 (empty index 0)
2019-12-10T22:32:27.491+05:30 [Info] ExhaustMove time: 0ns
2019-12-10T22:32:27.491+05:30 [Info] ExhaustMove call: 0
2019-12-10T22:32:27.491+05:30 [Info] ExhaustSwap time: 0ns
2019-12-10T22:32:27.491+05:30 [Info] ExhaustSwap call: 0
2019-12-10T22:32:27.491+05:30 [Info] ----------------------------------------
2019-12-10T22:32:27.491+05:30 [Info] 
2019-12-10T22:32:27.491+05:30 [Info] Indexer serverGroup:Group 1, nodeId:127.0.0.1:9001, nodeUUID:280ac5e1026b443deea667c0a12beceb, useLiveData:true
2019-12-10T22:32:27.491+05:30 [Info] Indexer total memory:1258291200 (1.17188G), mem:1258291200 (1.17188G), overhead:0 (0), min:1258306454 (1.17189G), data:1258367476 (1.17195G) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:3
2019-12-10T22:32:27.491+05:30 [Info] Indexer isDeleted:false isNew:false exclude: meetConstraint:true
2019-12-10T22:32:27.491+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:32:27.491+05:30 [Info] 		Index name:arr1 0, bucket:default, defnId:1619686062513106516, instId:8082202379049928859, Partition: 0, new/moved:false
2019-12-10T22:32:27.491+05:30 [Info] 		Index total memory:0 (0), mem:0 (0), overhead:0 (0), min:8316 (8.12109K), data:41583 (40.6084K) cpu:0.0000 io:0 (0) scan:0 drain:0
2019-12-10T22:32:27.491+05:30 [Info] 		Index resident:0% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:32:27.491+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:32:27.491+05:30 [Info] 		Index name:arr2 0, bucket:default, defnId:4823915220527584383, instId:11854930242797418660, Partition: 0, new/moved:false
2019-12-10T22:32:27.491+05:30 [Info] 		Index total memory:0 (0), mem:0 (0), overhead:0 (0), min:6938 (6.77539K), data:34693 (33.8799K) cpu:0.0000 io:0 (0) scan:0 drain:0
2019-12-10T22:32:27.491+05:30 [Info] 		Index resident:0% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:32:27.491+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:32:27.491+05:30 [Info] 		Index name:idx3 0, bucket:default, defnId:1085523998019490961, instId:11897793799436892494, Partition: 0, new/moved:true
2019-12-10T22:32:27.491+05:30 [Info] 		Index total memory:1258291200 (1.17188G), mem:1258291200 (1.17188G), overhead:0 (0), min:1258291200 (1.17188G), data:1258291200 (1.17188G) cpu:0.0000 io:0 (0) scan:0 drain:0
2019-12-10T22:32:27.491+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:32:27.491+05:30 [Info] ****************************************
2019-12-10T22:32:27.491+05:30 [Info] send commit create request to watcher 127.0.0.1:9106
2019-12-10T22:32:27.650+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:32:27.650+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:32:27.656+05:30 [Info] switched currmeta from 1344 -> 1346 force true 
2019-12-10T22:32:27.656+05:30 [Info] Rollback time has changed for index inst 11854930242797418660. New rollback time 1575996807857833855
2019-12-10T22:32:27.658+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:32:27.658+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:32:27.660+05:30 [Info] switched currmeta from 1340 -> 1342 force true 
2019-12-10T22:32:27.660+05:30 [Info] Rollback time has changed for index inst 11854930242797418660. New rollback time 1575996807857833855
2019-12-10T22:32:27.773+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:32:27.773+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:32:27.780+05:30 [Info] switched currmeta from 1342 -> 1343 force true 
2019-12-10T22:32:27.788+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:32:27.788+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:32:27.790+05:30 [Info] switched currmeta from 1346 -> 1347 force true 
2019-12-10T22:32:27.846+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:32:27.846+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:32:27.851+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:32:27.851+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:32:27.852+05:30 [Info] switched currmeta from 1347 -> 1348 force true 
2019-12-10T22:32:27.854+05:30 [Info] switched currmeta from 1343 -> 1344 force true 
2019-12-10T22:32:28.037+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:32:28.037+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:32:28.043+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:32:28.043+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:32:28.043+05:30 [Info] switched currmeta from 1344 -> 1345 force true 
2019-12-10T22:32:28.045+05:30 [Info] switched currmeta from 1348 -> 1349 force true 
2019-12-10T22:32:30.428+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:32:30.429+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:32:30.440+05:30 [Info] switched currmeta from 1345 -> 1346 force true 
2019-12-10T22:32:30.440+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:32:30.440+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:32:30.444+05:30 [Info] switched currmeta from 1349 -> 1350 force true 
2019-12-10T22:32:30.662+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:32:30.689+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:32:30.689+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:32:30.690+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:32:30.690+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:32:30.692+05:30 [Info] switched currmeta from 1346 -> 1346 force true 
2019-12-10T22:32:30.693+05:30 [Info] switched currmeta from 1350 -> 1350 force true 
2019-12-10T22:32:31.346+05:30 [Info] CreateIndex 1085523998019490961 default/idx3 using:forestdb exprType:N1QL whereExpr:() secExprs:([`name`]) desc:[] isPrimary:false scheme:SINGLE  partitionKeys:([]) with: - elapsed(4.37716234s) err()
2019/12/10 22:32:31 Created the secondary index idx3. Waiting for it become active
2019-12-10T22:32:31.346+05:30 [Info] metadata provider version changed 1350 -> 1351
2019-12-10T22:32:31.346+05:30 [Info] switched currmeta from 1350 -> 1351 force false 
2019/12/10 22:32:31 Index is now active
2019/12/10 22:32:31 Using n1ql client
2019-12-10T22:32:31.355+05:30 [Info] metadata provider version changed 1346 -> 1347
2019-12-10T22:32:31.355+05:30 [Info] switched currmeta from 1346 -> 1347 force false 
2019-12-10T22:32:31.355+05:30 [Info] GsiClient::UpdateUsecjson: using collatejson as data format between indexer and GsiClient
2019/12/10 22:32:31 Length of scanResults = 10
2019/12/10 22:32:31 DBG: ChangeIndexerSettings: nodes = [0xc01c2b0200]
2019/12/10 22:32:31 DBG: ChangeIndexerSettings: adminurl = 127.0.0.1:9106 host 127.0.0.1 sport 9106
2019/12/10 22:32:31 Changing config key indexer.settings.max_seckey_size to value 4096
2019/12/10 22:32:31 DBG: ChangeIndexerSettings: nodes = [0xc01c2b0700]
2019/12/10 22:32:31 DBG: ChangeIndexerSettings: adminurl = 127.0.0.1:9106 host 127.0.0.1 sport 9106
2019/12/10 22:32:31 Changing config key indexer.settings.max_array_seckey_size to value 51200
2019-12-10T22:32:31.520+05:30 [Info] New settings received: 
{"indexer.api.enableTestServer":true,"indexer.settings.allow_large_keys":false,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfFname":"","indexer.settings.cpuProfile":false,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":51200,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4096,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfFname":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":250,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.recovery.max_rollbacks":5,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":0,"indexer.settings.send_buffer_size":1024,"indexer.settings.sliceBufSize":50000,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"forestdb","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.wal_size":4096,"projector.settings.log_level":"info","queryport.client.settings.backfillLimit":0,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100}
2019-12-10T22:32:32.666+05:30 [Info] Rollback time has changed for index inst 5597714016550249329. New rollback time 1575996807857833855
2019-12-10T22:32:32.666+05:30 [Info] Rollback time has changed for index inst 5597714016550249329. New rollback time 1575996807857833855
2019-12-10T22:32:35.667+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:32:35.697+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:32:35.697+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:32:35.702+05:30 [Info] switched currmeta from 1351 -> 1351 force true 
2019-12-10T22:32:35.705+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:32:35.705+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:32:35.706+05:30 [Info] switched currmeta from 1347 -> 1347 force true 
2019-12-10T22:32:38.006+05:30 [Info] connected with 1 indexers
2019-12-10T22:32:38.006+05:30 [Info] index 1085523998019490961 has replicas [5597714016550249329]: 
2019-12-10T22:32:38.006+05:30 [Info] index 4823915220527584383 has replicas [11854930242797418660]: 
2019-12-10T22:32:38.006+05:30 [Info] index 1619686062513106516 has replicas [8082202379049928859]: 
2019-12-10T22:32:38.006+05:30 [Info] index 1085523998019490961 has equivalents [1085523998019490961]: 
2019-12-10T22:32:38.006+05:30 [Info] index 4823915220527584383 has equivalents [4823915220527584383]: 
2019-12-10T22:32:38.006+05:30 [Info] index 1619686062513106516 has equivalents [1619686062513106516]: 
2019-12-10T22:32:38.006+05:30 [Info] client load stats {"8082202379049928859": 0,"5597714016550249329": 752755,"11854930242797418660": 0}
2019-12-10T22:32:38.006+05:30 [Info] client hit stats {"11854930242797418660": 0,"8082202379049928859": 0,"5597714016550249329": 1}
2019-12-10T22:32:38.006+05:30 [Info] client pending item stats {"8082202379049928859": -9223372036854775699,"5597714016550249329": -9223372036854775699,"11854930242797418660": -9223372036854775699}
2019-12-10T22:32:38.006+05:30 [Info] client stats current {"5597714016550249329": true,"11854930242797418660": true,"8082202379049928859": true}
2019-12-10T22:32:38.009+05:30 [Info] num concurrent scans {0}
2019-12-10T22:32:38.009+05:30 [Info] average scan response {33 ms}
2019-12-10T22:32:40.667+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:32:40.690+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] active conns 0, free conns 1
2019-12-10T22:32:40.701+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:32:40.701+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:32:40.703+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:32:40.703+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:32:40.707+05:30 [Info] switched currmeta from 1347 -> 1347 force true 
2019-12-10T22:32:40.707+05:30 [Info] switched currmeta from 1351 -> 1351 force true 
2019/12/10 22:32:48 Expected and Actual scan responses are the same
2019/12/10 22:32:48 Using n1ql client
2019/12/10 22:32:48 Expected and Actual scan responses are the same
2019/12/10 22:32:48 Using n1ql client
2019/12/10 22:32:48 Expected and Actual scan responses are the same
2019/12/10 22:32:48 DBG: ChangeIndexerSettings: nodes = [0xc005fb3dc0]
2019/12/10 22:32:48 DBG: ChangeIndexerSettings: adminurl = 127.0.0.1:9106 host 127.0.0.1 sport 9106
2019/12/10 22:32:48 Changing config key indexer.settings.max_seckey_size to value 100
2019/12/10 22:32:48 DBG: ChangeIndexerSettings: nodes = [0xc014793800]
2019/12/10 22:32:48 DBG: ChangeIndexerSettings: adminurl = 127.0.0.1:9106 host 127.0.0.1 sport 9106
2019/12/10 22:32:48 Changing config key indexer.settings.max_array_seckey_size to value 2200
2019-12-10T22:32:48.709+05:30 [Info] New settings received: 
{"indexer.api.enableTestServer":true,"indexer.settings.allow_large_keys":false,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfFname":"","indexer.settings.cpuProfile":false,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":2200,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":100,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfFname":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":250,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.recovery.max_rollbacks":5,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":0,"indexer.settings.send_buffer_size":1024,"indexer.settings.sliceBufSize":50000,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"forestdb","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.wal_size":4096,"projector.settings.log_level":"info","queryport.client.settings.backfillLimit":0,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100}
2019-12-10T22:32:55.668+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:32:55.713+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:32:55.715+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:32:55.716+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:32:55.716+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:32:55.720+05:30 [Info] switched currmeta from 1351 -> 1351 force true 
2019-12-10T22:32:55.721+05:30 [Info] switched currmeta from 1347 -> 1347 force true 
2019-12-10T22:32:55.866+05:30 [Info] connected with 1 indexers
2019-12-10T22:32:55.866+05:30 [Info] index 4823915220527584383 has replicas [11854930242797418660]: 
2019-12-10T22:32:55.866+05:30 [Info] index 1085523998019490961 has replicas [5597714016550249329]: 
2019-12-10T22:32:55.866+05:30 [Info] index 1619686062513106516 has replicas [8082202379049928859]: 
2019-12-10T22:32:55.866+05:30 [Info] index 4823915220527584383 has equivalents [4823915220527584383]: 
2019-12-10T22:32:55.866+05:30 [Info] index 1085523998019490961 has equivalents [1085523998019490961]: 
2019-12-10T22:32:55.866+05:30 [Info] index 1619686062513106516 has equivalents [1619686062513106516]: 
2019-12-10T22:32:55.866+05:30 [Info] client load stats {"11854930242797418660": 2.5327115e+06,"5597714016550249329": 0,"8082202379049928859": 6.60442765275e+09}
2019-12-10T22:32:55.866+05:30 [Info] client hit stats {"11854930242797418660": 1,"5597714016550249329": 0,"8082202379049928859": 2}
2019-12-10T22:32:55.866+05:30 [Info] client pending item stats {"8082202379049928859": -9223372036854775699,"11854930242797418660": -9223372036854775699,"5597714016550249329": -9223372036854775699}
2019-12-10T22:32:55.866+05:30 [Info] client stats current {"11854930242797418660": true,"5597714016550249329": true,"8082202379049928859": true}
2019-12-10T22:32:55.874+05:30 [Info] num concurrent scans {1}
2019-12-10T22:32:55.874+05:30 [Info] average scan response {6608 ms}
2019-12-10T22:32:58.496+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] active conns 1, free conns 0
2019/12/10 22:33:08 Using n1ql client
2019/12/10 22:33:08 Length of scanResults = 10
2019/12/10 22:33:08 DBG: ChangeIndexerSettings: nodes = [0xc03c4ee940]
2019/12/10 22:33:08 DBG: ChangeIndexerSettings: adminurl = 127.0.0.1:9106 host 127.0.0.1 sport 9106
2019/12/10 22:33:08 Changing config key indexer.settings.max_seckey_size to value 4608
2019/12/10 22:33:08 DBG: ChangeIndexerSettings: nodes = [0xc03c4eea00]
2019/12/10 22:33:08 DBG: ChangeIndexerSettings: adminurl = 127.0.0.1:9106 host 127.0.0.1 sport 9106
2019/12/10 22:33:08 Changing config key indexer.settings.max_array_seckey_size to value 10240
2019-12-10T22:33:08.687+05:30 [Info] New settings received: 
{"indexer.api.enableTestServer":true,"indexer.settings.allow_large_keys":false,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfFname":"","indexer.settings.cpuProfile":false,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfFname":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":250,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.recovery.max_rollbacks":5,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":0,"indexer.settings.send_buffer_size":1024,"indexer.settings.sliceBufSize":50000,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"forestdb","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.wal_size":4096,"projector.settings.log_level":"info","queryport.client.settings.backfillLimit":0,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100}
--- PASS: TestArraySizeIncreaseDecrease1 (141.19s)
=== RUN   TestArraySizeIncreaseDecrease2
2019/12/10 22:33:09 In TestArraySizeIncreaseDecrease2()
Skip test as allow_large_keys = true is not supported for forestdb
--- PASS: TestArraySizeIncreaseDecrease2 (0.00s)
=== RUN   TestBufferedScan_BackfillDisabled
2019/12/10 22:33:09 In TestBufferedScan_BackfillDisabled()
2019/12/10 22:33:09 In DropAllSecondaryIndexes()
2019/12/10 22:33:09 Index found:  arr2
2019-12-10T22:33:09.801+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:33:09.801+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:33:09.802+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:33:09.802+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:33:09.804+05:30 [Info] switched currmeta from 1347 -> 1348 force true 
2019-12-10T22:33:09.805+05:30 [Info] switched currmeta from 1351 -> 1352 force true 
2019-12-10T22:33:09.862+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:33:09.862+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:33:09.864+05:30 [Info] metadata provider version changed 1352 -> 1353
2019-12-10T22:33:09.864+05:30 [Info] switched currmeta from 1352 -> 1353 force false 
2019-12-10T22:33:09.864+05:30 [Info] DropIndex 4823915220527584383 - elapsed(201.139525ms), err()
2019/12/10 22:33:09 Dropped index arr2
2019/12/10 22:33:09 Index found:  idx3
2019-12-10T22:33:09.864+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:33:09.864+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:33:09.871+05:30 [Info] switched currmeta from 1348 -> 1349 force true 
2019-12-10T22:33:09.883+05:30 [Info] switched currmeta from 1353 -> 1353 force true 
2019-12-10T22:33:09.964+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:33:09.964+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:33:09.972+05:30 [Info] switched currmeta from 1349 -> 1350 force true 
2019-12-10T22:33:09.992+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:33:09.992+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:33:09.995+05:30 [Info] switched currmeta from 1353 -> 1354 force true 
2019-12-10T22:33:10.040+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:33:10.040+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:33:10.045+05:30 [Info] switched currmeta from 1350 -> 1350 force true 
2019-12-10T22:33:10.077+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:33:10.077+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:33:10.080+05:30 [Info] switched currmeta from 1354 -> 1354 force true 
2019-12-10T22:33:10.101+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:33:10.101+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:33:10.105+05:30 [Info] switched currmeta from 1350 -> 1350 force true 
2019-12-10T22:33:10.108+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:33:10.108+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:33:10.111+05:30 [Info] metadata provider version changed 1354 -> 1355
2019-12-10T22:33:10.111+05:30 [Info] switched currmeta from 1354 -> 1355 force false 
2019-12-10T22:33:10.111+05:30 [Info] DropIndex 1085523998019490961 - elapsed(247.659531ms), err()
2019/12/10 22:33:10 Dropped index idx3
2019/12/10 22:33:10 Index found:  arr1
2019-12-10T22:33:10.123+05:30 [Info] switched currmeta from 1355 -> 1355 force true 
2019-12-10T22:33:10.254+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:33:10.254+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:33:10.263+05:30 [Info] switched currmeta from 1350 -> 1352 force true 
2019-12-10T22:33:10.272+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:33:10.272+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:33:10.307+05:30 [Info] switched currmeta from 1355 -> 1356 force true 
2019-12-10T22:33:10.359+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:33:10.359+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:33:10.365+05:30 [Info] switched currmeta from 1352 -> 1352 force true 
2019-12-10T22:33:10.371+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:33:10.371+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:33:10.386+05:30 [Info] switched currmeta from 1356 -> 1356 force true 
2019-12-10T22:33:10.428+05:30 [Info] metadata provider version changed 1356 -> 1357
2019-12-10T22:33:10.428+05:30 [Info] switched currmeta from 1356 -> 1357 force false 
2019-12-10T22:33:10.428+05:30 [Info] DropIndex 1619686062513106516 - elapsed(316.714665ms), err()
2019/12/10 22:33:10 Dropped index arr1
2019-12-10T22:33:10.462+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:33:10.462+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:33:10.478+05:30 [Info] switched currmeta from 1352 -> 1353 force true 
2019-12-10T22:33:10.531+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:33:10.531+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:33:10.543+05:30 [Info] switched currmeta from 1357 -> 1357 force true 
2019-12-10T22:33:10.682+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:33:10.682+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:33:10.699+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:33:10.699+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:33:10.706+05:30 [Info] switched currmeta from 1357 -> 1357 force true 
2019-12-10T22:33:10.713+05:30 [Info] switched currmeta from 1353 -> 1353 force true 
2019-12-10T22:33:11.340+05:30 [Info] GSIC[default/default-1575992531337069731] logstats "default" {"gsi_scan_count":77,"gsi_scan_duration":58574533354,"gsi_throttle_duration":576395961,"gsi_prime_duration":1788752120,"gsi_blocked_duration":465664674,"gsi_total_temp_files":0}
2019-12-10T22:33:12.553+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:33:12.590+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:33:12.590+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:33:12.593+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:33:12.594+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:33:12.595+05:30 [Info] switched currmeta from 1353 -> 1353 force true 
2019-12-10T22:33:12.598+05:30 [Info] switched currmeta from 1357 -> 1357 force true 
2019-12-10T22:33:15.400+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:33:15.448+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:33:15.448+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:33:15.455+05:30 [Info] switched currmeta from 1353 -> 1353 force true 
2019-12-10T22:33:15.462+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:33:15.462+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:33:15.464+05:30 [Info] switched currmeta from 1357 -> 1357 force true 
2019-12-10T22:33:15.724+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:33:15.754+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:33:15.754+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:33:15.757+05:30 [Info] switched currmeta from 1357 -> 1357 force true 
2019-12-10T22:33:15.761+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:33:15.761+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:33:15.763+05:30 [Info] switched currmeta from 1353 -> 1353 force true 
2019-12-10T22:33:20.399+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:33:20.447+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:33:20.447+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:33:20.451+05:30 [Info] switched currmeta from 1353 -> 1353 force true 
2019-12-10T22:33:20.453+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:33:20.453+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:33:20.456+05:30 [Info] switched currmeta from 1357 -> 1357 force true 
2019-12-10T22:33:22.553+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:33:22.620+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:33:22.620+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:33:22.620+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:33:22.620+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:33:22.622+05:30 [Info] switched currmeta from 1353 -> 1353 force true 
2019-12-10T22:33:22.624+05:30 [Info] switched currmeta from 1357 -> 1357 force true 
2019-12-10T22:33:38.006+05:30 [Info] connected with 1 indexers
2019-12-10T22:33:38.006+05:30 [Info] client load stats {}
2019-12-10T22:33:38.006+05:30 [Info] client hit stats {}
2019-12-10T22:33:38.006+05:30 [Info] client pending item stats {}
2019-12-10T22:33:38.006+05:30 [Info] client stats current {}
2019-12-10T22:33:38.009+05:30 [Info] num concurrent scans {0}
2019-12-10T22:33:38.009+05:30 [Info] average scan response {5 ms}
2019-12-10T22:33:40.708+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] active conns 0, free conns 1
2019-12-10T22:33:55.866+05:30 [Info] connected with 1 indexers
2019-12-10T22:33:55.866+05:30 [Info] client load stats {}
2019-12-10T22:33:55.866+05:30 [Info] client hit stats {}
2019-12-10T22:33:55.866+05:30 [Info] client pending item stats {}
2019-12-10T22:33:55.866+05:30 [Info] client stats current {}
2019-12-10T22:33:55.874+05:30 [Info] num concurrent scans {0}
2019-12-10T22:33:55.874+05:30 [Info] average scan response {5752 ms}
2019-12-10T22:33:58.513+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] active conns 0, free conns 1
2019/12/10 22:34:19 Flushed the bucket default
2019-12-10T22:34:38.006+05:30 [Info] connected with 1 indexers
2019-12-10T22:34:38.006+05:30 [Info] client load stats {}
2019-12-10T22:34:38.006+05:30 [Info] client hit stats {}
2019-12-10T22:34:38.006+05:30 [Info] client pending item stats {}
2019-12-10T22:34:38.006+05:30 [Info] client stats current {}
2019-12-10T22:34:38.009+05:30 [Info] num concurrent scans {0}
2019-12-10T22:34:38.009+05:30 [Info] average scan response {5 ms}
2019-12-10T22:34:40.733+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] active conns 0, free conns 1
2019-12-10T22:34:55.866+05:30 [Info] connected with 1 indexers
2019-12-10T22:34:55.866+05:30 [Info] client load stats {}
2019-12-10T22:34:55.866+05:30 [Info] client hit stats {}
2019-12-10T22:34:55.866+05:30 [Info] client pending item stats {}
2019-12-10T22:34:55.866+05:30 [Info] client stats current {}
2019-12-10T22:34:55.874+05:30 [Info] num concurrent scans {0}
2019-12-10T22:34:55.874+05:30 [Info] average scan response {5752 ms}
2019-12-10T22:34:58.545+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] active conns 0, free conns 1
2019-12-10T22:35:07.851+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9106 connected=true
2019/12/10 22:35:07 DBG: ChangeIndexerSettings: nodes = [0xc00abf20c0]
2019/12/10 22:35:07 DBG: ChangeIndexerSettings: adminurl = 127.0.0.1:9106 host 127.0.0.1 sport 9106
2019/12/10 22:35:07 Changing config key queryport.client.settings.backfillLimit to value 0
2019-12-10T22:35:07.870+05:30 [Info] send prepare create request to watcher 127.0.0.1:9106
2019-12-10T22:35:07.876+05:30 [Info] Indexer 127.0.0.1:9106 accept prepare request. Index (default, addressidx)
2019-12-10T22:35:07.976+05:30 [Info] Planner::processCreateToken: There is no create token to process for node 127.0.0.1:9001
2019-12-10T22:35:08.481+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node 127.0.0.1:9001
2019-12-10T22:35:08.566+05:30 [Info] ************ Indexer Layout *************
2019-12-10T22:35:08.566+05:30 [Info] Score: 0
2019-12-10T22:35:08.566+05:30 [Info] variation: 0
2019-12-10T22:35:08.567+05:30 [Info] ElapsedTime: 5ms
2019-12-10T22:35:08.567+05:30 [Info] ConvergenceTime: 3us
2019-12-10T22:35:08.567+05:30 [Info] Iteration: 0
2019-12-10T22:35:08.567+05:30 [Info] Move: 0
2019-12-10T22:35:08.567+05:30 [Info] ----------------------------------------
2019-12-10T22:35:08.567+05:30 [Info] Indexer Memory Mean 1258291200 (1.17188G)
2019-12-10T22:35:08.567+05:30 [Info] Indexer Memory Deviation 0 (0) (0.00%)
2019-12-10T22:35:08.567+05:30 [Info] Indexer Memory Utilization 0.8000
2019-12-10T22:35:08.567+05:30 [Info] Indexer CPU Mean 0.0000
2019-12-10T22:35:08.567+05:30 [Info] Indexer CPU Deviation 0.00 (0.00%)
2019-12-10T22:35:08.567+05:30 [Info] Indexer CPU Utilization 0.0000
2019-12-10T22:35:08.567+05:30 [Info] Indexer IO Mean 0.0000
2019-12-10T22:35:08.567+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2019-12-10T22:35:08.567+05:30 [Info] Indexer Drain Rate Mean 0.0000
2019-12-10T22:35:08.567+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2019-12-10T22:35:08.567+05:30 [Info] Indexer Scan Rate Mean 0.0000
2019-12-10T22:35:08.567+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2019-12-10T22:35:08.567+05:30 [Info] Indexer Data Size Mean 1258291200 (1.17188G)
2019-12-10T22:35:08.567+05:30 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2019-12-10T22:35:08.567+05:30 [Info] Total Index Data (from non-deleted node) 0
2019-12-10T22:35:08.567+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2019-12-10T22:35:08.567+05:30 [Info] No. Index (from non-deleted node) 0
2019-12-10T22:35:08.567+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2019-12-10T22:35:08.567+05:30 [Info] ----------------------------------------
2019-12-10T22:35:08.567+05:30 [Info] Number of indexes: 1
2019-12-10T22:35:08.567+05:30 [Info] Number of indexers: 1
2019-12-10T22:35:08.567+05:30 [Info] Avg Index Size: 1258291200 (1.17188G)
2019-12-10T22:35:08.567+05:30 [Info] Max Index Size: 1258291200 (1.17188G)
2019-12-10T22:35:08.567+05:30 [Info] Max Indexer Overhead: 0 (0)
2019-12-10T22:35:08.567+05:30 [Info] Avg Index Cpu: 0.0000
2019-12-10T22:35:08.567+05:30 [Info] Max Index Cpu: 0.0000
2019-12-10T22:35:08.567+05:30 [Info] Num Estimation Run: 1
2019-12-10T22:35:08.567+05:30 [Info] ----------------------------------------
2019-12-10T22:35:08.567+05:30 [Info] Memory Quota 1572864000 (1.46484G)
2019-12-10T22:35:08.567+05:30 [Info] CPU Quota 4
2019-12-10T22:35:08.567+05:30 [Info] Max Cpu Utilization -1
2019-12-10T22:35:08.567+05:30 [Info] Max Memory Utilization -1
2019-12-10T22:35:08.567+05:30 [Info] ----------------------------------------
2019-12-10T22:35:08.567+05:30 [Info] Total iteration: 0
2019-12-10T22:35:08.567+05:30 [Info] RandomSwap time: 0ns
2019-12-10T22:35:08.567+05:30 [Info] RandomSwap call: 0
2019-12-10T22:35:08.567+05:30 [Info] RandomSwap iteration: 0
2019-12-10T22:35:08.567+05:30 [Info] RandomMove time: 0ns
2019-12-10T22:35:08.567+05:30 [Info] RandomMove call: 0 (empty index 0)
2019-12-10T22:35:08.567+05:30 [Info] ExhaustMove time: 0ns
2019-12-10T22:35:08.567+05:30 [Info] ExhaustMove call: 0
2019-12-10T22:35:08.567+05:30 [Info] ExhaustSwap time: 0ns
2019-12-10T22:35:08.568+05:30 [Info] ExhaustSwap call: 0
2019-12-10T22:35:08.568+05:30 [Info] ----------------------------------------
2019-12-10T22:35:08.568+05:30 [Info] 
2019-12-10T22:35:08.568+05:30 [Info] Indexer serverGroup:Group 1, nodeId:127.0.0.1:9001, nodeUUID:280ac5e1026b443deea667c0a12beceb, useLiveData:true
2019-12-10T22:35:08.568+05:30 [Info] Indexer total memory:1258291200 (1.17188G), mem:1258291200 (1.17188G), overhead:0 (0), min:1258291200 (1.17188G), data:1258291200 (1.17188G) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:1
2019-12-10T22:35:08.568+05:30 [Info] Indexer isDeleted:false isNew:false exclude: meetConstraint:true
2019-12-10T22:35:08.568+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:35:08.568+05:30 [Info] 		Index name:addressidx 0, bucket:default, defnId:9453058082312876815, instId:5616238083526960117, Partition: 0, new/moved:true
2019-12-10T22:35:08.568+05:30 [Info] 		Index total memory:1258291200 (1.17188G), mem:1258291200 (1.17188G), overhead:0 (0), min:1258291200 (1.17188G), data:1258291200 (1.17188G) cpu:0.0000 io:0 (0) scan:0 drain:0
2019-12-10T22:35:08.568+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:35:08.568+05:30 [Info] ****************************************
2019-12-10T22:35:08.568+05:30 [Info] send commit create request to watcher 127.0.0.1:9106
2019-12-10T22:35:08.810+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:35:08.811+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:35:08.817+05:30 [Info] switched currmeta from 1353 -> 1355 force true 
2019-12-10T22:35:08.823+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:35:08.823+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:35:08.837+05:30 [Info] switched currmeta from 1357 -> 1359 force true 
2019-12-10T22:35:09.008+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:35:09.008+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:35:09.010+05:30 [Info] switched currmeta from 1359 -> 1360 force true 
2019-12-10T22:35:09.014+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:35:09.014+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:35:09.019+05:30 [Info] switched currmeta from 1355 -> 1356 force true 
2019-12-10T22:35:09.091+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:35:09.091+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:35:09.092+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:35:09.092+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:35:09.101+05:30 [Info] switched currmeta from 1360 -> 1361 force true 
2019-12-10T22:35:09.101+05:30 [Info] switched currmeta from 1356 -> 1357 force true 
2019-12-10T22:35:09.313+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:35:09.313+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:35:09.321+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:35:09.321+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:35:09.323+05:30 [Info] switched currmeta from 1357 -> 1358 force true 
2019-12-10T22:35:09.326+05:30 [Info] switched currmeta from 1361 -> 1362 force true 
2019-12-10T22:35:12.668+05:30 [Info] Rollback time has changed for index inst 18069853558395000838. New rollback time 1575996807857833855
2019-12-10T22:35:12.669+05:30 [Info] Rollback time has changed for index inst 18069853558395000838. New rollback time 1575996807857833855
2019-12-10T22:35:15.749+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:35:15.785+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:35:15.785+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:35:15.790+05:30 [Info] switched currmeta from 1362 -> 1362 force true 
2019-12-10T22:35:15.795+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:35:15.795+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:35:15.797+05:30 [Info] switched currmeta from 1358 -> 1358 force true 
2019-12-10T22:35:16.705+05:30 [Info] CreateIndex 9453058082312876815 default/addressidx using:forestdb exprType:N1QL whereExpr:() secExprs:([`address`]) desc:[] isPrimary:false scheme:SINGLE  partitionKeys:([]) with: - elapsed(8.836034597s) err()
2019/12/10 22:35:16 Created the secondary index addressidx. Waiting for it become active
2019-12-10T22:35:16.705+05:30 [Info] metadata provider version changed 1362 -> 1363
2019-12-10T22:35:16.706+05:30 [Info] switched currmeta from 1362 -> 1363 force false 
2019/12/10 22:35:16 Index is now active
2019-12-10T22:35:16.711+05:30 [Info] metadata provider version changed 1358 -> 1359
2019-12-10T22:35:16.711+05:30 [Info] switched currmeta from 1358 -> 1359 force false 
2019-12-10T22:35:16.711+05:30 [Info] GsiClient::UpdateUsecjson: using collatejson as data format between indexer and GsiClient
2019-12-10T22:35:16.711+05:30 [Info] GSIC[default/default-1575997516706057165] started ...
2019/12/10 22:35:16 Non-backfill file found: /tmp/.ICE-unix
2019/12/10 22:35:16 Non-backfill file found: /tmp/.Test-unix
2019/12/10 22:35:16 Non-backfill file found: /tmp/.X11-unix
2019/12/10 22:35:16 Non-backfill file found: /tmp/.XIM-unix
2019/12/10 22:35:16 Non-backfill file found: /tmp/.font-unix
2019/12/10 22:35:16 Non-backfill file found: /tmp/fail.log
2019/12/10 22:35:16 Non-backfill file found: /tmp/go-build099006873
2019/12/10 22:35:16 Non-backfill file found: /tmp/go-build873429666
2019/12/10 22:35:16 Non-backfill file found: /tmp/logdir
2019/12/10 22:35:16 Non-backfill file found: /tmp/mdbslice
2019/12/10 22:35:16 Non-backfill file found: /tmp/systemd-private-7c2e2c27ccea48c98c55fd7a7e7f28d7-apache2.service-Kou1Rz
2019/12/10 22:35:16 Non-backfill file found: /tmp/systemd-private-7c2e2c27ccea48c98c55fd7a7e7f28d7-systemd-timesyncd.service-QUvJF6
2019/12/10 22:35:16 limit=1,chsize=256; received 1 items; took 3.314817ms
2019-12-10T22:35:16.750+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:35:16.750+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:35:16.750+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:35:16.750+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:35:16.754+05:30 [Info] switched currmeta from 1363 -> 1363 force true 
2019-12-10T22:35:16.754+05:30 [Info] switched currmeta from 1359 -> 1359 force true 
2019/12/10 22:35:18 limit=1000,chsize=256; received 1000 items; took 1.26299113s
--- PASS: TestBufferedScan_BackfillDisabled (130.32s)
=== RUN   TestBufferedScan_BackfillEnabled
2019/12/10 22:35:19 In TestBufferedScan_BackfillEnabled()
2019-12-10T22:35:19.980+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9106 connected=true
2019/12/10 22:35:19 DBG: ChangeIndexerSettings: nodes = [0xc077d0c840]
2019/12/10 22:35:19 DBG: ChangeIndexerSettings: adminurl = 127.0.0.1:9106 host 127.0.0.1 sport 9106
2019/12/10 22:35:19 Changing config key queryport.client.settings.backfillLimit to value 1
2019-12-10T22:35:20.005+05:30 [Info] GsiClient::UpdateUsecjson: using collatejson as data format between indexer and GsiClient
2019-12-10T22:35:20.006+05:30 [Info] GSIC[default/default-1575997519990396767] started ...
2019-12-10T22:35:20.031+05:30 [Info] New settings received: 
{"indexer.api.enableTestServer":true,"indexer.settings.allow_large_keys":false,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfFname":"","indexer.settings.cpuProfile":false,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfFname":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":250,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.recovery.max_rollbacks":5,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":0,"indexer.settings.send_buffer_size":1024,"indexer.settings.sliceBufSize":50000,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"forestdb","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.wal_size":4096,"projector.settings.log_level":"info","queryport.client.settings.backfillLimit":1,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100}
2019/12/10 22:35:20 limit=1,chsize=256; received 1 items; took 54.560923ms
2019-12-10T22:35:20.750+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:35:20.776+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:35:20.776+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:35:20.778+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:35:20.778+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:35:20.780+05:30 [Info] switched currmeta from 1359 -> 1359 force true 
2019-12-10T22:35:20.781+05:30 [Info] switched currmeta from 1363 -> 1363 force true 
2019/12/10 22:35:21 limit=1000,chsize=256; received 1000 items; took 22.229064ms
2019-12-10T22:35:22.096+05:30 [Info] GSIC[default/default-1575997519990396767] bufferedscan new temp file ... /tmp/scan-results22760819527077
2019-12-10T22:35:25.751+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:35:25.780+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:35:25.780+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:35:25.780+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:35:25.781+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:35:25.783+05:30 [Info] switched currmeta from 1363 -> 1363 force true 
2019-12-10T22:35:25.784+05:30 [Info] switched currmeta from 1359 -> 1359 force true 
2019-12-10T22:35:28.068+05:30 [Info] GSIC[default/default-1575997519990396767] request(bufferedscan) removing temp file /tmp/scan-results22760819527077 ...
2019/12/10 22:35:33 limit=1000,chsize=256; received 1000 items; took 10.23674268s
2019-12-10T22:35:34.324+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] open new connection ...
2019-12-10T22:35:34.338+05:30 [Info] GSIC[default/default-1575997519990396767] bufferedscan new temp file ... /tmp/scan-results22760292409448
2019-12-10T22:35:34.352+05:30 [Info] GSIC[default/default-1575997519990396767] bufferedscan new temp file ... /tmp/scan-results22760779361754
Scan error: "bufferedscan" temp file size exceeded limit 1, 8 - cause: "bufferedscan" temp file size exceeded limit 1, 8
Scan error: "bufferedscan" temp file size exceeded limit 1, 8 - cause: "bufferedscan" temp file size exceeded limit 1, 8
Scan error:  "bufferedscan" temp file size exceeded limit 1, 8 - cause:  "bufferedscan" temp file size exceeded limit 1, 8
2019-12-10T22:35:35.787+05:30 [Info] GSIC[default/default-1575997519990396767] request(bufferedscan) removing temp file /tmp/scan-results22760292409448 ...
Scan error:  "bufferedscan" temp file size exceeded limit 1, 8 - cause:  "bufferedscan" temp file size exceeded limit 1, 8
2019-12-10T22:35:35.788+05:30 [Info] GSIC[default/default-1575997519990396767] request(bufferedscan) removing temp file /tmp/scan-results22760779361754 ...
2019-12-10T22:35:38.006+05:30 [Info] connected with 1 indexers
2019-12-10T22:35:38.006+05:30 [Info] index 9453058082312876815 has replicas [18069853558395000838]: 
2019-12-10T22:35:38.006+05:30 [Info] index 9453058082312876815 has equivalents [9453058082312876815]: 
2019-12-10T22:35:38.006+05:30 [Info] client load stats {"18069853558395000838": 5.29090964484375e+08}
2019-12-10T22:35:38.006+05:30 [Info] client hit stats {"18069853558395000838": 7}
2019-12-10T22:35:38.006+05:30 [Info] client pending item stats {"18069853558395000838": 9223372036854775807}
2019-12-10T22:35:38.006+05:30 [Info] client stats current {"18069853558395000838": true}
2019-12-10T22:35:38.009+05:30 [Info] num concurrent scans {0}
2019-12-10T22:35:38.009+05:30 [Info] average scan response {3042 ms}
2019-12-10T22:35:40.753+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:35:40.768+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] active conns 0, free conns 2
2019-12-10T22:35:40.783+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:35:40.783+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:35:40.786+05:30 [Info] switched currmeta from 1359 -> 1359 force true 
2019-12-10T22:35:40.787+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:35:40.787+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:35:40.789+05:30 [Info] switched currmeta from 1363 -> 1363 force true 
2019/12/10 22:35:46 limit=1000,chsize=256; received 585 items; took 11.847389595s
2019/12/10 22:35:46 limit=1000,chsize=256; received 585 items; took 11.847366308s
2019-12-10T22:35:47.171+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9106 connected=true
2019/12/10 22:35:47 DBG: ChangeIndexerSettings: nodes = [0xc02ee664c0]
2019/12/10 22:35:47 DBG: ChangeIndexerSettings: adminurl = 127.0.0.1:9106 host 127.0.0.1 sport 9106
2019/12/10 22:35:47 Changing config key queryport.client.settings.backfillLimit to value 0
--- PASS: TestBufferedScan_BackfillEnabled (27.20s)
=== RUN   TestMultiScanSetup
2019/12/10 22:35:47 In TestMultiScanSetup()
2019-12-10T22:35:47.203+05:30 [Info] New settings received: 
{"indexer.api.enableTestServer":true,"indexer.settings.allow_large_keys":false,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfFname":"","indexer.settings.cpuProfile":false,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfFname":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":250,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.recovery.max_rollbacks":5,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":0,"indexer.settings.send_buffer_size":1024,"indexer.settings.sliceBufSize":50000,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"forestdb","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.wal_size":4096,"projector.settings.log_level":"info","queryport.client.settings.backfillLimit":0,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100}
2019/12/10 22:35:48 Emptying the default bucket
2019/12/10 22:35:51 Flush Enabled on bucket default
2019-12-10T22:35:52.553+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:35:52.621+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:35:52.621+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:35:52.622+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:35:52.622+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:35:52.629+05:30 [Info] switched currmeta from 1359 -> 1359 force true 
2019-12-10T22:35:52.630+05:30 [Info] switched currmeta from 1363 -> 1363 force true 
2019-12-10T22:35:55.431+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:35:55.485+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:35:55.485+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:35:55.494+05:30 [Info] switched currmeta from 1359 -> 1359 force true 
2019-12-10T22:35:55.498+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:35:55.498+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:35:55.501+05:30 [Info] switched currmeta from 1363 -> 1363 force true 
2019-12-10T22:35:55.866+05:30 [Info] connected with 1 indexers
2019-12-10T22:35:55.866+05:30 [Info] index 9453058082312876815 has replicas [18069853558395000838]: 
2019-12-10T22:35:55.866+05:30 [Info] index 9453058082312876815 has equivalents [9453058082312876815]: 
2019-12-10T22:35:55.866+05:30 [Info] client load stats {"18069853558395000838": 0}
2019-12-10T22:35:55.866+05:30 [Info] client hit stats {"18069853558395000838": 0}
2019-12-10T22:35:55.866+05:30 [Info] client pending item stats {"18069853558395000838": 9223372036854775807}
2019-12-10T22:35:55.866+05:30 [Info] client stats current {"18069853558395000838": true}
2019-12-10T22:35:55.874+05:30 [Info] num concurrent scans {0}
2019-12-10T22:35:55.874+05:30 [Info] average scan response {5752 ms}
2019-12-10T22:35:58.576+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] active conns 0, free conns 1
2019-12-10T22:36:00.397+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:36:00.444+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:36:00.444+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:36:00.446+05:30 [Info] switched currmeta from 1359 -> 1359 force true 
2019-12-10T22:36:00.458+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:36:00.458+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:36:00.466+05:30 [Info] switched currmeta from 1363 -> 1363 force true 
2019-12-10T22:36:02.556+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:36:02.607+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:36:02.607+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:36:02.613+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:36:02.613+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:36:02.614+05:30 [Info] switched currmeta from 1363 -> 1363 force true 
2019-12-10T22:36:02.617+05:30 [Info] switched currmeta from 1359 -> 1359 force true 
2019-12-10T22:36:07.664+05:30 [Info] Rollback time has changed for index inst 18069853558395000838. New rollback time 1575997565336307514
2019-12-10T22:36:07.664+05:30 [Info] Rollback time has changed for index inst 18069853558395000838. New rollback time 1575997565336307514
2019-12-10T22:36:12.642+05:30 [Info] Rollback time has changed for index inst 18069853558395000838. New rollback time 1575997571077922096
2019-12-10T22:36:12.644+05:30 [Info] Rollback time has changed for index inst 18069853558395000838. New rollback time 1575997571077922096
2019-12-10T22:36:38.006+05:30 [Info] connected with 1 indexers
2019-12-10T22:36:38.006+05:30 [Info] index 9453058082312876815 has replicas [18069853558395000838]: 
2019-12-10T22:36:38.006+05:30 [Info] index 9453058082312876815 has equivalents [9453058082312876815]: 
2019-12-10T22:36:38.006+05:30 [Info] client load stats {"18069853558395000838": 5.29090964484375e+08}
2019-12-10T22:36:38.006+05:30 [Info] client hit stats {"18069853558395000838": 7}
2019-12-10T22:36:38.006+05:30 [Info] client pending item stats {"18069853558395000838": 9223372036854775807}
2019-12-10T22:36:38.006+05:30 [Info] client stats current {"18069853558395000838": true}
2019-12-10T22:36:38.009+05:30 [Info] num concurrent scans {0}
2019-12-10T22:36:38.009+05:30 [Info] average scan response {3042 ms}
2019-12-10T22:36:40.786+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] active conns 0, free conns 2
2019-12-10T22:36:55.866+05:30 [Info] connected with 1 indexers
2019-12-10T22:36:55.866+05:30 [Info] index 9453058082312876815 has replicas [18069853558395000838]: 
2019-12-10T22:36:55.866+05:30 [Info] index 9453058082312876815 has equivalents [9453058082312876815]: 
2019-12-10T22:36:55.866+05:30 [Info] client load stats {"18069853558395000838": 0}
2019-12-10T22:36:55.866+05:30 [Info] client hit stats {"18069853558395000838": 0}
2019-12-10T22:36:55.866+05:30 [Info] client pending item stats {"18069853558395000838": 9223372036854775807}
2019-12-10T22:36:55.866+05:30 [Info] client stats current {"18069853558395000838": true}
2019-12-10T22:36:55.874+05:30 [Info] num concurrent scans {0}
2019-12-10T22:36:55.874+05:30 [Info] average scan response {5752 ms}
2019-12-10T22:36:58.593+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] active conns 0, free conns 1
2019/12/10 22:36:59 Flushed the bucket default
2019/12/10 22:37:04 Populating the default bucket
2019-12-10T22:37:12.963+05:30 [Info] send prepare create request to watcher 127.0.0.1:9106
2019-12-10T22:37:12.963+05:30 [Info] Indexer 127.0.0.1:9106 accept prepare request. Index (default, index_companyname)
2019-12-10T22:37:13.037+05:30 [Info] Planner::processCreateToken: There is no create token to process for node 127.0.0.1:9001
2019-12-10T22:37:13.500+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node 127.0.0.1:9001
2019-12-10T22:37:13.591+05:30 [Info] ************ Indexer Layout *************
2019-12-10T22:37:13.592+05:30 [Info] Score: 0
2019-12-10T22:37:13.592+05:30 [Info] variation: 0
2019-12-10T22:37:13.592+05:30 [Info] ElapsedTime: 8ms
2019-12-10T22:37:13.592+05:30 [Info] ConvergenceTime: 2us
2019-12-10T22:37:13.592+05:30 [Info] Iteration: 0
2019-12-10T22:37:13.592+05:30 [Info] Move: 0
2019-12-10T22:37:13.592+05:30 [Info] ----------------------------------------
2019-12-10T22:37:13.592+05:30 [Info] Indexer Memory Mean 1258291200 (1.17188G)
2019-12-10T22:37:13.592+05:30 [Info] Indexer Memory Deviation 0 (0) (0.00%)
2019-12-10T22:37:13.592+05:30 [Info] Indexer Memory Utilization 0.8000
2019-12-10T22:37:13.592+05:30 [Info] Indexer CPU Mean 0.0138
2019-12-10T22:37:13.592+05:30 [Info] Indexer CPU Deviation 0.00 (0.00%)
2019-12-10T22:37:13.592+05:30 [Info] Indexer CPU Utilization 0.0034
2019-12-10T22:37:13.592+05:30 [Info] Indexer IO Mean 250570.0000
2019-12-10T22:37:13.592+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2019-12-10T22:37:13.592+05:30 [Info] Indexer Drain Rate Mean 331.0000
2019-12-10T22:37:13.592+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2019-12-10T22:37:13.592+05:30 [Info] Indexer Scan Rate Mean 517.0000
2019-12-10T22:37:13.592+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2019-12-10T22:37:13.592+05:30 [Info] Indexer Data Size Mean 1263202297 (1.17645G)
2019-12-10T22:37:13.592+05:30 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2019-12-10T22:37:13.592+05:30 [Info] Total Index Data (from non-deleted node) 0
2019-12-10T22:37:13.592+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2019-12-10T22:37:13.592+05:30 [Info] No. Index (from non-deleted node) 0
2019-12-10T22:37:13.592+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2019-12-10T22:37:13.592+05:30 [Info] ----------------------------------------
2019-12-10T22:37:13.592+05:30 [Info] Number of indexes: 2
2019-12-10T22:37:13.593+05:30 [Info] Number of indexers: 1
2019-12-10T22:37:13.593+05:30 [Info] Avg Index Size: 629145600 (600M)
2019-12-10T22:37:13.593+05:30 [Info] Max Index Size: 1258291200 (1.17188G)
2019-12-10T22:37:13.593+05:30 [Info] Max Indexer Overhead: 0 (0)
2019-12-10T22:37:13.593+05:30 [Info] Avg Index Cpu: 0.0069
2019-12-10T22:37:13.593+05:30 [Info] Max Index Cpu: 0.0138
2019-12-10T22:37:13.593+05:30 [Info] Num Estimation Run: 1
2019-12-10T22:37:13.593+05:30 [Info] ----------------------------------------
2019-12-10T22:37:13.593+05:30 [Info] Memory Quota 1572864000 (1.46484G)
2019-12-10T22:37:13.593+05:30 [Info] CPU Quota 4
2019-12-10T22:37:13.593+05:30 [Info] Max Cpu Utilization -1
2019-12-10T22:37:13.593+05:30 [Info] Max Memory Utilization -1
2019-12-10T22:37:13.593+05:30 [Info] ----------------------------------------
2019-12-10T22:37:13.593+05:30 [Info] Total iteration: 0
2019-12-10T22:37:13.593+05:30 [Info] RandomSwap time: 0ns
2019-12-10T22:37:13.593+05:30 [Info] RandomSwap call: 0
2019-12-10T22:37:13.593+05:30 [Info] RandomSwap iteration: 0
2019-12-10T22:37:13.593+05:30 [Info] RandomMove time: 0ns
2019-12-10T22:37:13.593+05:30 [Info] RandomMove call: 0 (empty index 0)
2019-12-10T22:37:13.593+05:30 [Info] ExhaustMove time: 0ns
2019-12-10T22:37:13.593+05:30 [Info] ExhaustMove call: 0
2019-12-10T22:37:13.593+05:30 [Info] ExhaustSwap time: 0ns
2019-12-10T22:37:13.593+05:30 [Info] ExhaustSwap call: 0
2019-12-10T22:37:13.593+05:30 [Info] ----------------------------------------
2019-12-10T22:37:13.593+05:30 [Info] 
2019-12-10T22:37:13.593+05:30 [Info] Indexer serverGroup:Group 1, nodeId:127.0.0.1:9001, nodeUUID:280ac5e1026b443deea667c0a12beceb, useLiveData:true
2019-12-10T22:37:13.593+05:30 [Info] Indexer total memory:1258291200 (1.17188G), mem:1258291200 (1.17188G), overhead:0 (0), min:1259273419 (1.17279G), data:1263202297 (1.17645G) cpu:0.0138, io:250570 (244.697K), scan:517 drain:331 numIndexes:2
2019-12-10T22:37:13.593+05:30 [Info] Indexer isDeleted:false isNew:false exclude: meetConstraint:true
2019-12-10T22:37:13.593+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:37:13.593+05:30 [Info] 		Index name:addressidx 0, bucket:default, defnId:9453058082312876815, instId:18069853558395000838, Partition: 0, new/moved:false
2019-12-10T22:37:13.593+05:30 [Info] 		Index total memory:0 (0), mem:0 (0), overhead:0 (0), min:982219 (959.198K), data:4911097 (4.68359M) cpu:0.0138 io:250570 (244.697K) scan:517 drain:331
2019-12-10T22:37:13.593+05:30 [Info] 		Index resident:0% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:37:13.593+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:37:13.593+05:30 [Info] 		Index name:index_companyname 0, bucket:default, defnId:13362352446024974117, instId:5327948734403176264, Partition: 0, new/moved:true
2019-12-10T22:37:13.593+05:30 [Info] 		Index total memory:1258291200 (1.17188G), mem:1258291200 (1.17188G), overhead:0 (0), min:1258291200 (1.17188G), data:1258291200 (1.17188G) cpu:0.0000 io:0 (0) scan:0 drain:0
2019-12-10T22:37:13.593+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:37:13.593+05:30 [Info] ****************************************
2019-12-10T22:37:13.593+05:30 [Info] send commit create request to watcher 127.0.0.1:9106
2019-12-10T22:37:13.821+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:13.821+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:13.827+05:30 [Info] switched currmeta from 1359 -> 1361 force true 
2019-12-10T22:37:13.835+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:13.835+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:13.837+05:30 [Info] switched currmeta from 1363 -> 1365 force true 
2019-12-10T22:37:14.046+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:14.046+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:14.051+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:14.051+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:14.052+05:30 [Info] switched currmeta from 1365 -> 1366 force true 
2019-12-10T22:37:14.055+05:30 [Info] switched currmeta from 1361 -> 1362 force true 
2019-12-10T22:37:14.101+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:14.101+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:14.104+05:30 [Info] switched currmeta from 1366 -> 1367 force true 
2019-12-10T22:37:14.108+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:14.108+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:14.112+05:30 [Info] switched currmeta from 1362 -> 1363 force true 
2019-12-10T22:37:14.312+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:14.312+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:14.314+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:14.314+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:14.316+05:30 [Info] switched currmeta from 1363 -> 1364 force true 
2019-12-10T22:37:14.317+05:30 [Info] switched currmeta from 1367 -> 1368 force true 
2019-12-10T22:37:17.720+05:30 [Info] Rollback time has changed for index inst 8926027219439387868. New rollback time 1575997571077922096
2019-12-10T22:37:17.720+05:30 [Info] Rollback time has changed for index inst 8926027219439387868. New rollback time 1575997571077922096
2019-12-10T22:37:18.196+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:18.196+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:18.203+05:30 [Info] switched currmeta from 1364 -> 1365 force true 
2019-12-10T22:37:18.206+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:18.206+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:18.210+05:30 [Info] switched currmeta from 1368 -> 1369 force true 
2019-12-10T22:37:18.806+05:30 [Info] CreateIndex 13362352446024974117 default/index_companyname using:forestdb exprType:N1QL whereExpr:() secExprs:([`company` `name`]) desc:[] isPrimary:false scheme:SINGLE  partitionKeys:([]) with: - elapsed(5.843417515s) err()
2019/12/10 22:37:18 Created the secondary index index_companyname. Waiting for it become active
2019-12-10T22:37:18.806+05:30 [Info] metadata provider version changed 1369 -> 1370
2019-12-10T22:37:18.806+05:30 [Info] switched currmeta from 1369 -> 1370 force false 
2019/12/10 22:37:18 Index is now active
2019-12-10T22:37:18.806+05:30 [Info] send prepare create request to watcher 127.0.0.1:9106
2019-12-10T22:37:18.811+05:30 [Info] Indexer 127.0.0.1:9106 accept prepare request. Index (default, index_company)
2019-12-10T22:37:18.876+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:18.877+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:18.878+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:18.878+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:18.885+05:30 [Info] switched currmeta from 1370 -> 1370 force true 
2019-12-10T22:37:18.888+05:30 [Info] switched currmeta from 1365 -> 1366 force true 
2019-12-10T22:37:18.940+05:30 [Info] Planner::processCreateToken: There is no create token to process for node 127.0.0.1:9001
2019-12-10T22:37:19.300+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node 127.0.0.1:9001
2019-12-10T22:37:19.386+05:30 [Info] ************ Indexer Layout *************
2019-12-10T22:37:19.386+05:30 [Info] Score: 0
2019-12-10T22:37:19.386+05:30 [Info] variation: 0
2019-12-10T22:37:19.386+05:30 [Info] ElapsedTime: 8ms
2019-12-10T22:37:19.386+05:30 [Info] ConvergenceTime: 2us
2019-12-10T22:37:19.386+05:30 [Info] Iteration: 0
2019-12-10T22:37:19.386+05:30 [Info] Move: 0
2019-12-10T22:37:19.386+05:30 [Info] ----------------------------------------
2019-12-10T22:37:19.386+05:30 [Info] Indexer Memory Mean 1258291200 (1.17188G)
2019-12-10T22:37:19.386+05:30 [Info] Indexer Memory Deviation 0 (0) (0.00%)
2019-12-10T22:37:19.386+05:30 [Info] Indexer Memory Utilization 0.8000
2019-12-10T22:37:19.386+05:30 [Info] Indexer CPU Mean 0.0138
2019-12-10T22:37:19.386+05:30 [Info] Indexer CPU Deviation 0.00 (0.00%)
2019-12-10T22:37:19.386+05:30 [Info] Indexer CPU Utilization 0.0034
2019-12-10T22:37:19.386+05:30 [Info] Indexer IO Mean 152228.0000
2019-12-10T22:37:19.386+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2019-12-10T22:37:19.386+05:30 [Info] Indexer Drain Rate Mean 246.0000
2019-12-10T22:37:19.386+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2019-12-10T22:37:19.386+05:30 [Info] Indexer Scan Rate Mean 258.0000
2019-12-10T22:37:19.386+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2019-12-10T22:37:19.386+05:30 [Info] Indexer Data Size Mean 1266748035 (1.17975G)
2019-12-10T22:37:19.386+05:30 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2019-12-10T22:37:19.386+05:30 [Info] Total Index Data (from non-deleted node) 0
2019-12-10T22:37:19.386+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2019-12-10T22:37:19.386+05:30 [Info] No. Index (from non-deleted node) 0
2019-12-10T22:37:19.386+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2019-12-10T22:37:19.386+05:30 [Info] ----------------------------------------
2019-12-10T22:37:19.386+05:30 [Info] Number of indexes: 3
2019-12-10T22:37:19.386+05:30 [Info] Number of indexers: 1
2019-12-10T22:37:19.386+05:30 [Info] Avg Index Size: 419430400 (400M)
2019-12-10T22:37:19.386+05:30 [Info] Max Index Size: 1258291200 (1.17188G)
2019-12-10T22:37:19.386+05:30 [Info] Max Indexer Overhead: 0 (0)
2019-12-10T22:37:19.386+05:30 [Info] Avg Index Cpu: 0.0046
2019-12-10T22:37:19.386+05:30 [Info] Max Index Cpu: 0.0138
2019-12-10T22:37:19.386+05:30 [Info] Num Estimation Run: 1
2019-12-10T22:37:19.386+05:30 [Info] ----------------------------------------
2019-12-10T22:37:19.386+05:30 [Info] Memory Quota 1572864000 (1.46484G)
2019-12-10T22:37:19.386+05:30 [Info] CPU Quota 4
2019-12-10T22:37:19.386+05:30 [Info] Max Cpu Utilization -1
2019-12-10T22:37:19.386+05:30 [Info] Max Memory Utilization -1
2019-12-10T22:37:19.386+05:30 [Info] ----------------------------------------
2019-12-10T22:37:19.386+05:30 [Info] Total iteration: 0
2019-12-10T22:37:19.386+05:30 [Info] RandomSwap time: 0ns
2019-12-10T22:37:19.386+05:30 [Info] RandomSwap call: 0
2019-12-10T22:37:19.386+05:30 [Info] RandomSwap iteration: 0
2019-12-10T22:37:19.386+05:30 [Info] RandomMove time: 0ns
2019-12-10T22:37:19.386+05:30 [Info] RandomMove call: 0 (empty index 0)
2019-12-10T22:37:19.386+05:30 [Info] ExhaustMove time: 0ns
2019-12-10T22:37:19.386+05:30 [Info] ExhaustMove call: 0
2019-12-10T22:37:19.386+05:30 [Info] ExhaustSwap time: 0ns
2019-12-10T22:37:19.386+05:30 [Info] ExhaustSwap call: 0
2019-12-10T22:37:19.386+05:30 [Info] ----------------------------------------
2019-12-10T22:37:19.386+05:30 [Info] 
2019-12-10T22:37:19.386+05:30 [Info] Indexer serverGroup:Group 1, nodeId:127.0.0.1:9001, nodeUUID:280ac5e1026b443deea667c0a12beceb, useLiveData:true
2019-12-10T22:37:19.386+05:30 [Info] Indexer total memory:1258291200 (1.17188G), mem:1258291200 (1.17188G), overhead:0 (0), min:1259982566 (1.17345G), data:1266748035 (1.17975G) cpu:0.0138, io:152228 (148.66K), scan:258 drain:246 numIndexes:3
2019-12-10T22:37:19.386+05:30 [Info] Indexer isDeleted:false isNew:false exclude: meetConstraint:true
2019-12-10T22:37:19.386+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:37:19.386+05:30 [Info] 		Index name:addressidx 0, bucket:default, defnId:9453058082312876815, instId:18069853558395000838, Partition: 0, new/moved:false
2019-12-10T22:37:19.386+05:30 [Info] 		Index total memory:0 (0), mem:0 (0), overhead:0 (0), min:1009522 (985.861K), data:5047611 (4.81378M) cpu:0.0138 io:152228 (148.66K) scan:258 drain:246
2019-12-10T22:37:19.386+05:30 [Info] 		Index resident:0% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:37:19.386+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:37:19.386+05:30 [Info] 		Index name:index_companyname 0, bucket:default, defnId:13362352446024974117, instId:8926027219439387868, Partition: 0, new/moved:false
2019-12-10T22:37:19.386+05:30 [Info] 		Index total memory:0 (0), mem:0 (0), overhead:0 (0), min:681844 (665.863K), data:3409224 (3.25129M) cpu:0.0000 io:0 (0) scan:0 drain:0
2019-12-10T22:37:19.386+05:30 [Info] 		Index resident:0% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:37:19.386+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:37:19.386+05:30 [Info] 		Index name:index_company 0, bucket:default, defnId:6560167570622610970, instId:1441071905881724192, Partition: 0, new/moved:true
2019-12-10T22:37:19.386+05:30 [Info] 		Index total memory:1258291200 (1.17188G), mem:1258291200 (1.17188G), overhead:0 (0), min:1258291200 (1.17188G), data:1258291200 (1.17188G) cpu:0.0000 io:0 (0) scan:0 drain:0
2019-12-10T22:37:19.386+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:37:19.386+05:30 [Info] ****************************************
2019-12-10T22:37:19.387+05:30 [Info] send commit create request to watcher 127.0.0.1:9106
2019-12-10T22:37:19.540+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:19.540+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:19.543+05:30 [Info] switched currmeta from 1366 -> 1368 force true 
2019-12-10T22:37:19.552+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:19.552+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:19.555+05:30 [Info] switched currmeta from 1370 -> 1372 force true 
2019-12-10T22:37:19.684+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:19.684+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:19.685+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:19.685+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:19.691+05:30 [Info] switched currmeta from 1368 -> 1369 force true 
2019-12-10T22:37:19.693+05:30 [Info] switched currmeta from 1372 -> 1373 force true 
2019-12-10T22:37:19.751+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:19.751+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:19.755+05:30 [Info] switched currmeta from 1369 -> 1370 force true 
2019-12-10T22:37:19.758+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:19.758+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:19.760+05:30 [Info] switched currmeta from 1373 -> 1374 force true 
2019-12-10T22:37:19.912+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:19.912+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:19.921+05:30 [Info] switched currmeta from 1374 -> 1375 force true 
2019-12-10T22:37:19.923+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:19.923+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:19.925+05:30 [Info] switched currmeta from 1370 -> 1371 force true 
2019-12-10T22:37:20.817+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:37:20.916+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:20.916+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:20.928+05:30 [Info] switched currmeta from 1375 -> 1375 force true 
2019-12-10T22:37:20.967+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:20.967+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:20.975+05:30 [Info] switched currmeta from 1371 -> 1371 force true 
2019-12-10T22:37:23.411+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:23.411+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:23.412+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:23.412+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:23.415+05:30 [Info] switched currmeta from 1371 -> 1372 force true 
2019-12-10T22:37:23.416+05:30 [Info] switched currmeta from 1375 -> 1376 force true 
2019-12-10T22:37:23.931+05:30 [Info] CreateIndex 6560167570622610970 default/index_company using:forestdb exprType:N1QL whereExpr:() secExprs:([`company`]) desc:[] isPrimary:false scheme:SINGLE  partitionKeys:([]) with: - elapsed(5.12479592s) err()
2019/12/10 22:37:23 Created the secondary index index_company. Waiting for it become active
2019-12-10T22:37:23.931+05:30 [Info] metadata provider version changed 1376 -> 1377
2019-12-10T22:37:23.931+05:30 [Info] switched currmeta from 1376 -> 1377 force false 
2019/12/10 22:37:23 Index is now active
2019-12-10T22:37:23.932+05:30 [Info] send prepare create request to watcher 127.0.0.1:9106
2019-12-10T22:37:23.940+05:30 [Info] Indexer 127.0.0.1:9106 accept prepare request. Index (default, index_company_name_age)
2019-12-10T22:37:24.017+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:24.017+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:24.024+05:30 [Info] switched currmeta from 1372 -> 1373 force true 
2019-12-10T22:37:24.025+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:24.025+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:24.028+05:30 [Info] switched currmeta from 1377 -> 1377 force true 
2019-12-10T22:37:24.065+05:30 [Info] Planner::processCreateToken: There is no create token to process for node 127.0.0.1:9001
2019-12-10T22:37:24.396+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node 127.0.0.1:9001
2019-12-10T22:37:24.490+05:30 [Info] ************ Indexer Layout *************
2019-12-10T22:37:24.490+05:30 [Info] Score: 0
2019-12-10T22:37:24.490+05:30 [Info] variation: 0
2019-12-10T22:37:24.490+05:30 [Info] ElapsedTime: 11ms
2019-12-10T22:37:24.490+05:30 [Info] ConvergenceTime: 2us
2019-12-10T22:37:24.490+05:30 [Info] Iteration: 0
2019-12-10T22:37:24.490+05:30 [Info] Move: 0
2019-12-10T22:37:24.490+05:30 [Info] ----------------------------------------
2019-12-10T22:37:24.490+05:30 [Info] Indexer Memory Mean 1258291200 (1.17188G)
2019-12-10T22:37:24.490+05:30 [Info] Indexer Memory Deviation 0 (0) (0.00%)
2019-12-10T22:37:24.490+05:30 [Info] Indexer Memory Utilization 0.8000
2019-12-10T22:37:24.490+05:30 [Info] Indexer CPU Mean 0.0138
2019-12-10T22:37:24.490+05:30 [Info] Indexer CPU Deviation 0.00 (0.00%)
2019-12-10T22:37:24.490+05:30 [Info] Indexer CPU Utilization 0.0034
2019-12-10T22:37:24.490+05:30 [Info] Indexer IO Mean 152228.0000
2019-12-10T22:37:24.490+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2019-12-10T22:37:24.490+05:30 [Info] Indexer Drain Rate Mean 246.0000
2019-12-10T22:37:24.490+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2019-12-10T22:37:24.490+05:30 [Info] Indexer Scan Rate Mean 258.0000
2019-12-10T22:37:24.490+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2019-12-10T22:37:24.490+05:30 [Info] Indexer Data Size Mean 1269838295 (1.18263G)
2019-12-10T22:37:24.490+05:30 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2019-12-10T22:37:24.490+05:30 [Info] Total Index Data (from non-deleted node) 0
2019-12-10T22:37:24.490+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2019-12-10T22:37:24.490+05:30 [Info] No. Index (from non-deleted node) 0
2019-12-10T22:37:24.490+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2019-12-10T22:37:24.490+05:30 [Info] ----------------------------------------
2019-12-10T22:37:24.490+05:30 [Info] Number of indexes: 4
2019-12-10T22:37:24.490+05:30 [Info] Number of indexers: 1
2019-12-10T22:37:24.490+05:30 [Info] Avg Index Size: 314572800 (300M)
2019-12-10T22:37:24.490+05:30 [Info] Max Index Size: 1258291200 (1.17188G)
2019-12-10T22:37:24.490+05:30 [Info] Max Indexer Overhead: 0 (0)
2019-12-10T22:37:24.490+05:30 [Info] Avg Index Cpu: 0.0034
2019-12-10T22:37:24.491+05:30 [Info] Max Index Cpu: 0.0138
2019-12-10T22:37:24.491+05:30 [Info] Num Estimation Run: 1
2019-12-10T22:37:24.491+05:30 [Info] ----------------------------------------
2019-12-10T22:37:24.491+05:30 [Info] Memory Quota 1572864000 (1.46484G)
2019-12-10T22:37:24.491+05:30 [Info] CPU Quota 4
2019-12-10T22:37:24.491+05:30 [Info] Max Cpu Utilization -1
2019-12-10T22:37:24.491+05:30 [Info] Max Memory Utilization -1
2019-12-10T22:37:24.491+05:30 [Info] ----------------------------------------
2019-12-10T22:37:24.491+05:30 [Info] Total iteration: 0
2019-12-10T22:37:24.491+05:30 [Info] RandomSwap time: 0ns
2019-12-10T22:37:24.491+05:30 [Info] RandomSwap call: 0
2019-12-10T22:37:24.491+05:30 [Info] RandomSwap iteration: 0
2019-12-10T22:37:24.491+05:30 [Info] RandomMove time: 0ns
2019-12-10T22:37:24.491+05:30 [Info] RandomMove call: 0 (empty index 0)
2019-12-10T22:37:24.491+05:30 [Info] ExhaustMove time: 0ns
2019-12-10T22:37:24.491+05:30 [Info] ExhaustMove call: 0
2019-12-10T22:37:24.491+05:30 [Info] ExhaustSwap time: 0ns
2019-12-10T22:37:24.491+05:30 [Info] ExhaustSwap call: 0
2019-12-10T22:37:24.491+05:30 [Info] ----------------------------------------
2019-12-10T22:37:24.491+05:30 [Info] 
2019-12-10T22:37:24.491+05:30 [Info] Indexer serverGroup:Group 1, nodeId:127.0.0.1:9001, nodeUUID:280ac5e1026b443deea667c0a12beceb, useLiveData:true
2019-12-10T22:37:24.491+05:30 [Info] Indexer total memory:1258291200 (1.17188G), mem:1258291200 (1.17188G), overhead:0 (0), min:1260600618 (1.17403G), data:1269838295 (1.18263G) cpu:0.0138, io:152228 (148.66K), scan:258 drain:246 numIndexes:4
2019-12-10T22:37:24.491+05:30 [Info] Indexer isDeleted:false isNew:false exclude: meetConstraint:true
2019-12-10T22:37:24.491+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:37:24.491+05:30 [Info] 		Index name:addressidx 0, bucket:default, defnId:9453058082312876815, instId:18069853558395000838, Partition: 0, new/moved:false
2019-12-10T22:37:24.491+05:30 [Info] 		Index total memory:0 (0), mem:0 (0), overhead:0 (0), min:1009522 (985.861K), data:5047611 (4.81378M) cpu:0.0138 io:152228 (148.66K) scan:258 drain:246
2019-12-10T22:37:24.491+05:30 [Info] 		Index resident:0% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:37:24.491+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:37:24.491+05:30 [Info] 		Index name:index_companyname 0, bucket:default, defnId:13362352446024974117, instId:8926027219439387868, Partition: 0, new/moved:false
2019-12-10T22:37:24.491+05:30 [Info] 		Index total memory:0 (0), mem:0 (0), overhead:0 (0), min:681844 (665.863K), data:3409224 (3.25129M) cpu:0.0000 io:0 (0) scan:0 drain:0
2019-12-10T22:37:24.491+05:30 [Info] 		Index resident:0% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:37:24.491+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:37:24.491+05:30 [Info] 		Index name:index_company 0, bucket:default, defnId:6560167570622610970, instId:14131000415694366248, Partition: 0, new/moved:false
2019-12-10T22:37:24.491+05:30 [Info] 		Index total memory:0 (0), mem:0 (0), overhead:0 (0), min:618052 (603.566K), data:3090260 (2.9471M) cpu:0.0000 io:0 (0) scan:0 drain:0
2019-12-10T22:37:24.491+05:30 [Info] 		Index resident:0% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:37:24.491+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:37:24.491+05:30 [Info] 		Index name:index_company_name_age 0, bucket:default, defnId:16166739134347805642, instId:7635682626257458635, Partition: 0, new/moved:true
2019-12-10T22:37:24.491+05:30 [Info] 		Index total memory:1258291200 (1.17188G), mem:1258291200 (1.17188G), overhead:0 (0), min:1258291200 (1.17188G), data:1258291200 (1.17188G) cpu:0.0000 io:0 (0) scan:0 drain:0
2019-12-10T22:37:24.491+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:37:24.491+05:30 [Info] ****************************************
2019-12-10T22:37:24.491+05:30 [Info] send commit create request to watcher 127.0.0.1:9106
2019-12-10T22:37:24.630+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:24.630+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:24.633+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:24.633+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:24.635+05:30 [Info] switched currmeta from 1373 -> 1375 force true 
2019-12-10T22:37:24.637+05:30 [Info] switched currmeta from 1377 -> 1379 force true 
2019-12-10T22:37:24.769+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:24.769+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:24.774+05:30 [Info] switched currmeta from 1379 -> 1380 force true 
2019-12-10T22:37:24.774+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:24.774+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:24.779+05:30 [Info] switched currmeta from 1375 -> 1376 force true 
2019-12-10T22:37:24.821+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:24.821+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:24.826+05:30 [Info] switched currmeta from 1380 -> 1381 force true 
2019-12-10T22:37:24.834+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:24.834+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:24.837+05:30 [Info] switched currmeta from 1376 -> 1377 force true 
2019-12-10T22:37:24.972+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:24.972+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:24.983+05:30 [Info] switched currmeta from 1377 -> 1378 force true 
2019-12-10T22:37:24.985+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:24.985+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:24.992+05:30 [Info] switched currmeta from 1381 -> 1382 force true 
2019-12-10T22:37:25.820+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:37:25.927+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:25.927+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:25.932+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:25.932+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:25.935+05:30 [Info] switched currmeta from 1378 -> 1378 force true 
2019-12-10T22:37:25.936+05:30 [Info] switched currmeta from 1382 -> 1382 force true 
2019-12-10T22:37:27.681+05:30 [Info] Rollback time has changed for index inst 14131000415694366248. New rollback time 1575997571077922096
2019-12-10T22:37:27.682+05:30 [Info] Rollback time has changed for index inst 14131000415694366248. New rollback time 1575997571077922096
2019-12-10T22:37:28.838+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:28.838+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:28.842+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:28.842+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:28.845+05:30 [Info] switched currmeta from 1382 -> 1383 force true 
2019-12-10T22:37:28.849+05:30 [Info] switched currmeta from 1378 -> 1379 force true 
2019-12-10T22:37:29.389+05:30 [Info] CreateIndex 16166739134347805642 default/index_company_name_age using:forestdb exprType:N1QL whereExpr:() secExprs:([`company` `name` `age`]) desc:[] isPrimary:false scheme:SINGLE  partitionKeys:([]) with: - elapsed(5.457725419s) err()
2019/12/10 22:37:29 Created the secondary index index_company_name_age. Waiting for it become active
2019-12-10T22:37:29.389+05:30 [Info] metadata provider version changed 1383 -> 1384
2019-12-10T22:37:29.389+05:30 [Info] switched currmeta from 1383 -> 1384 force false 
2019/12/10 22:37:29 Index is now active
2019-12-10T22:37:29.389+05:30 [Info] send prepare create request to watcher 127.0.0.1:9106
2019-12-10T22:37:29.395+05:30 [Info] Indexer 127.0.0.1:9106 accept prepare request. Index (default, index_primary)
2019-12-10T22:37:29.488+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:29.488+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:29.494+05:30 [Info] switched currmeta from 1384 -> 1384 force true 
2019-12-10T22:37:29.497+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:29.497+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:29.502+05:30 [Info] switched currmeta from 1379 -> 1380 force true 
2019-12-10T22:37:29.548+05:30 [Info] Planner::processCreateToken: There is no create token to process for node 127.0.0.1:9001
2019-12-10T22:37:29.864+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node 127.0.0.1:9001
2019-12-10T22:37:30.032+05:30 [Info] ************ Indexer Layout *************
2019-12-10T22:37:30.032+05:30 [Info] Score: 0
2019-12-10T22:37:30.032+05:30 [Info] variation: 0
2019-12-10T22:37:30.032+05:30 [Info] ElapsedTime: 22ms
2019-12-10T22:37:30.032+05:30 [Info] ConvergenceTime: 3us
2019-12-10T22:37:30.032+05:30 [Info] Iteration: 0
2019-12-10T22:37:30.032+05:30 [Info] Move: 0
2019-12-10T22:37:30.032+05:30 [Info] ----------------------------------------
2019-12-10T22:37:30.032+05:30 [Info] Indexer Memory Mean 1258291200 (1.17188G)
2019-12-10T22:37:30.032+05:30 [Info] Indexer Memory Deviation 0 (0) (0.00%)
2019-12-10T22:37:30.032+05:30 [Info] Indexer Memory Utilization 0.8000
2019-12-10T22:37:30.032+05:30 [Info] Indexer CPU Mean 0.2136
2019-12-10T22:37:30.032+05:30 [Info] Indexer CPU Deviation 0.00 (0.00%)
2019-12-10T22:37:30.032+05:30 [Info] Indexer CPU Utilization 0.0534
2019-12-10T22:37:30.032+05:30 [Info] Indexer IO Mean 152228.0000
2019-12-10T22:37:30.033+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2019-12-10T22:37:30.033+05:30 [Info] Indexer Drain Rate Mean 246.0000
2019-12-10T22:37:30.033+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2019-12-10T22:37:30.033+05:30 [Info] Indexer Scan Rate Mean 258.0000
2019-12-10T22:37:30.033+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2019-12-10T22:37:30.033+05:30 [Info] Indexer Data Size Mean 1273408689 (1.18595G)
2019-12-10T22:37:30.033+05:30 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2019-12-10T22:37:30.033+05:30 [Info] Total Index Data (from non-deleted node) 0
2019-12-10T22:37:30.033+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2019-12-10T22:37:30.033+05:30 [Info] No. Index (from non-deleted node) 0
2019-12-10T22:37:30.033+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2019-12-10T22:37:30.033+05:30 [Info] ----------------------------------------
2019-12-10T22:37:30.033+05:30 [Info] Number of indexes: 5
2019-12-10T22:37:30.033+05:30 [Info] Number of indexers: 1
2019-12-10T22:37:30.033+05:30 [Info] Avg Index Size: 251658240 (240M)
2019-12-10T22:37:30.033+05:30 [Info] Max Index Size: 1258291200 (1.17188G)
2019-12-10T22:37:30.033+05:30 [Info] Max Indexer Overhead: 0 (0)
2019-12-10T22:37:30.033+05:30 [Info] Avg Index Cpu: 0.0427
2019-12-10T22:37:30.033+05:30 [Info] Max Index Cpu: 0.2136
2019-12-10T22:37:30.033+05:30 [Info] Num Estimation Run: 1
2019-12-10T22:37:30.033+05:30 [Info] ----------------------------------------
2019-12-10T22:37:30.033+05:30 [Info] Memory Quota 1572864000 (1.46484G)
2019-12-10T22:37:30.033+05:30 [Info] CPU Quota 4
2019-12-10T22:37:30.033+05:30 [Info] Max Cpu Utilization -1
2019-12-10T22:37:30.033+05:30 [Info] Max Memory Utilization -1
2019-12-10T22:37:30.033+05:30 [Info] ----------------------------------------
2019-12-10T22:37:30.033+05:30 [Info] Total iteration: 0
2019-12-10T22:37:30.033+05:30 [Info] RandomSwap time: 0ns
2019-12-10T22:37:30.033+05:30 [Info] RandomSwap call: 0
2019-12-10T22:37:30.033+05:30 [Info] RandomSwap iteration: 0
2019-12-10T22:37:30.033+05:30 [Info] RandomMove time: 0ns
2019-12-10T22:37:30.033+05:30 [Info] RandomMove call: 0 (empty index 0)
2019-12-10T22:37:30.033+05:30 [Info] ExhaustMove time: 0ns
2019-12-10T22:37:30.033+05:30 [Info] ExhaustMove call: 0
2019-12-10T22:37:30.033+05:30 [Info] ExhaustSwap time: 0ns
2019-12-10T22:37:30.033+05:30 [Info] ExhaustSwap call: 0
2019-12-10T22:37:30.033+05:30 [Info] ----------------------------------------
2019-12-10T22:37:30.033+05:30 [Info] 
2019-12-10T22:37:30.033+05:30 [Info] Indexer serverGroup:Group 1, nodeId:127.0.0.1:9001, nodeUUID:280ac5e1026b443deea667c0a12beceb, useLiveData:true
2019-12-10T22:37:30.033+05:30 [Info] Indexer total memory:1258291200 (1.17188G), mem:1258291200 (1.17188G), overhead:0 (0), min:1261314696 (1.17469G), data:1273408689 (1.18595G) cpu:0.2136, io:152228 (148.66K), scan:258 drain:246 numIndexes:5
2019-12-10T22:37:30.033+05:30 [Info] Indexer isDeleted:false isNew:false exclude: meetConstraint:true
2019-12-10T22:37:30.033+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:37:30.033+05:30 [Info] 		Index name:addressidx 0, bucket:default, defnId:9453058082312876815, instId:18069853558395000838, Partition: 0, new/moved:false
2019-12-10T22:37:30.033+05:30 [Info] 		Index total memory:0 (0), mem:0 (0), overhead:0 (0), min:1009522 (985.861K), data:5047611 (4.81378M) cpu:0.2136 io:152228 (148.66K) scan:258 drain:246
2019-12-10T22:37:30.033+05:30 [Info] 		Index resident:0% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:37:30.033+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:37:30.033+05:30 [Info] 		Index name:index_companyname 0, bucket:default, defnId:13362352446024974117, instId:8926027219439387868, Partition: 0, new/moved:false
2019-12-10T22:37:30.033+05:30 [Info] 		Index total memory:0 (0), mem:0 (0), overhead:0 (0), min:681844 (665.863K), data:3409224 (3.25129M) cpu:0.0000 io:0 (0) scan:0 drain:0
2019-12-10T22:37:30.033+05:30 [Info] 		Index resident:0% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:37:30.033+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:37:30.033+05:30 [Info] 		Index name:index_company 0, bucket:default, defnId:6560167570622610970, instId:14131000415694366248, Partition: 0, new/moved:false
2019-12-10T22:37:30.034+05:30 [Info] 		Index total memory:0 (0), mem:0 (0), overhead:0 (0), min:618052 (603.566K), data:3090260 (2.9471M) cpu:0.0000 io:0 (0) scan:0 drain:0
2019-12-10T22:37:30.034+05:30 [Info] 		Index resident:0% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:37:30.034+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:37:30.034+05:30 [Info] 		Index name:index_company_name_age 0, bucket:default, defnId:16166739134347805642, instId:13682299663548290129, Partition: 0, new/moved:false
2019-12-10T22:37:30.034+05:30 [Info] 		Index total memory:0 (0), mem:0 (0), overhead:0 (0), min:714078 (697.342K), data:3570394 (3.40499M) cpu:0.0000 io:0 (0) scan:0 drain:0
2019-12-10T22:37:30.034+05:30 [Info] 		Index resident:0% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:37:30.034+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:37:30.034+05:30 [Info] 		Index name:index_primary 0, bucket:default, defnId:18131217113229780221, instId:16054817173315676933, Partition: 0, new/moved:true
2019-12-10T22:37:30.034+05:30 [Info] 		Index total memory:1258291200 (1.17188G), mem:1258291200 (1.17188G), overhead:0 (0), min:1258291200 (1.17188G), data:1258291200 (1.17188G) cpu:0.0000 io:0 (0) scan:0 drain:0
2019-12-10T22:37:30.034+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:37:30.034+05:30 [Info] ****************************************
2019-12-10T22:37:30.034+05:30 [Info] send commit create request to watcher 127.0.0.1:9106
2019-12-10T22:37:30.191+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:30.191+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:30.194+05:30 [Info] switched currmeta from 1384 -> 1386 force true 
2019-12-10T22:37:30.195+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:30.195+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:30.198+05:30 [Info] switched currmeta from 1380 -> 1382 force true 
2019-12-10T22:37:30.334+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:30.334+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:30.336+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:30.337+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:30.342+05:30 [Info] switched currmeta from 1386 -> 1388 force true 
2019-12-10T22:37:30.344+05:30 [Info] switched currmeta from 1382 -> 1384 force true 
2019-12-10T22:37:30.389+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:30.389+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:30.392+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:30.392+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:30.392+05:30 [Info] switched currmeta from 1388 -> 1388 force true 
2019-12-10T22:37:30.396+05:30 [Info] switched currmeta from 1384 -> 1384 force true 
2019-12-10T22:37:30.568+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:30.568+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:30.568+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:30.568+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:30.571+05:30 [Info] switched currmeta from 1388 -> 1389 force true 
2019-12-10T22:37:30.571+05:30 [Info] switched currmeta from 1384 -> 1385 force true 
2019-12-10T22:37:30.827+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:37:30.883+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:30.883+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:30.886+05:30 [Info] switched currmeta from 1389 -> 1389 force true 
2019-12-10T22:37:30.891+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:30.891+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:30.894+05:30 [Info] switched currmeta from 1385 -> 1385 force true 
2019-12-10T22:37:32.744+05:30 [Info] Rollback time has changed for index inst 13682299663548290129. New rollback time 1575997571077922096
2019-12-10T22:37:32.744+05:30 [Info] Rollback time has changed for index inst 13682299663548290129. New rollback time 1575997571077922096
2019-12-10T22:37:33.825+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:33.825+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:33.827+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:33.827+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:33.830+05:30 [Info] switched currmeta from 1385 -> 1386 force true 
2019-12-10T22:37:33.830+05:30 [Info] switched currmeta from 1389 -> 1390 force true 
2019-12-10T22:37:34.434+05:30 [Info] CreateIndex 18131217113229780221 default/index_primary using:forestdb exprType:N1QL whereExpr:() secExprs:([]) desc:[] isPrimary:true scheme:SINGLE  partitionKeys:([]) with: - elapsed(5.044635371s) err()
2019/12/10 22:37:34 Created the secondary index index_primary. Waiting for it become active
2019-12-10T22:37:34.434+05:30 [Info] metadata provider version changed 1390 -> 1391
2019-12-10T22:37:34.434+05:30 [Info] switched currmeta from 1390 -> 1391 force false 
2019/12/10 22:37:34 Index is now active
2019-12-10T22:37:34.435+05:30 [Info] send prepare create request to watcher 127.0.0.1:9106
2019-12-10T22:37:34.437+05:30 [Info] Indexer 127.0.0.1:9106 accept prepare request. Index (default, index_company_name_age_address)
2019-12-10T22:37:34.525+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:34.525+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:34.531+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:34.531+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:34.532+05:30 [Info] switched currmeta from 1386 -> 1387 force true 
2019-12-10T22:37:34.540+05:30 [Info] switched currmeta from 1391 -> 1391 force true 
2019-12-10T22:37:34.595+05:30 [Info] Planner::processCreateToken: There is no create token to process for node 127.0.0.1:9001
2019-12-10T22:37:34.935+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node 127.0.0.1:9001
2019-12-10T22:37:35.041+05:30 [Info] ************ Indexer Layout *************
2019-12-10T22:37:35.041+05:30 [Info] Score: 0
2019-12-10T22:37:35.041+05:30 [Info] variation: 0
2019-12-10T22:37:35.041+05:30 [Info] ElapsedTime: 15ms
2019-12-10T22:37:35.041+05:30 [Info] ConvergenceTime: 4us
2019-12-10T22:37:35.041+05:30 [Info] Iteration: 0
2019-12-10T22:37:35.041+05:30 [Info] Move: 0
2019-12-10T22:37:35.041+05:30 [Info] ----------------------------------------
2019-12-10T22:37:35.041+05:30 [Info] Indexer Memory Mean 1258291200 (1.17188G)
2019-12-10T22:37:35.041+05:30 [Info] Indexer Memory Deviation 0 (0) (0.00%)
2019-12-10T22:37:35.041+05:30 [Info] Indexer Memory Utilization 0.8000
2019-12-10T22:37:35.041+05:30 [Info] Indexer CPU Mean 0.2136
2019-12-10T22:37:35.041+05:30 [Info] Indexer CPU Deviation 0.00 (0.00%)
2019-12-10T22:37:35.041+05:30 [Info] Indexer CPU Utilization 0.0534
2019-12-10T22:37:35.041+05:30 [Info] Indexer IO Mean 152228.0000
2019-12-10T22:37:35.041+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2019-12-10T22:37:35.041+05:30 [Info] Indexer Drain Rate Mean 246.0000
2019-12-10T22:37:35.041+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2019-12-10T22:37:35.041+05:30 [Info] Indexer Scan Rate Mean 258.0000
2019-12-10T22:37:35.041+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2019-12-10T22:37:35.041+05:30 [Info] Indexer Data Size Mean 1274533309 (1.187G)
2019-12-10T22:37:35.041+05:30 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2019-12-10T22:37:35.041+05:30 [Info] Total Index Data (from non-deleted node) 0
2019-12-10T22:37:35.041+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2019-12-10T22:37:35.041+05:30 [Info] No. Index (from non-deleted node) 0
2019-12-10T22:37:35.041+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2019-12-10T22:37:35.041+05:30 [Info] ----------------------------------------
2019-12-10T22:37:35.041+05:30 [Info] Number of indexes: 6
2019-12-10T22:37:35.041+05:30 [Info] Number of indexers: 1
2019-12-10T22:37:35.041+05:30 [Info] Avg Index Size: 209715200 (200M)
2019-12-10T22:37:35.041+05:30 [Info] Max Index Size: 1258291200 (1.17188G)
2019-12-10T22:37:35.041+05:30 [Info] Max Indexer Overhead: 0 (0)
2019-12-10T22:37:35.041+05:30 [Info] Avg Index Cpu: 0.0356
2019-12-10T22:37:35.041+05:30 [Info] Max Index Cpu: 0.2136
2019-12-10T22:37:35.041+05:30 [Info] Num Estimation Run: 1
2019-12-10T22:37:35.041+05:30 [Info] ----------------------------------------
2019-12-10T22:37:35.041+05:30 [Info] Memory Quota 1572864000 (1.46484G)
2019-12-10T22:37:35.041+05:30 [Info] CPU Quota 4
2019-12-10T22:37:35.041+05:30 [Info] Max Cpu Utilization -1
2019-12-10T22:37:35.041+05:30 [Info] Max Memory Utilization -1
2019-12-10T22:37:35.041+05:30 [Info] ----------------------------------------
2019-12-10T22:37:35.041+05:30 [Info] Total iteration: 0
2019-12-10T22:37:35.041+05:30 [Info] RandomSwap time: 0ns
2019-12-10T22:37:35.041+05:30 [Info] RandomSwap call: 0
2019-12-10T22:37:35.041+05:30 [Info] RandomSwap iteration: 0
2019-12-10T22:37:35.041+05:30 [Info] RandomMove time: 0ns
2019-12-10T22:37:35.041+05:30 [Info] RandomMove call: 0 (empty index 0)
2019-12-10T22:37:35.041+05:30 [Info] ExhaustMove time: 0ns
2019-12-10T22:37:35.041+05:30 [Info] ExhaustMove call: 0
2019-12-10T22:37:35.041+05:30 [Info] ExhaustSwap time: 0ns
2019-12-10T22:37:35.041+05:30 [Info] ExhaustSwap call: 0
2019-12-10T22:37:35.041+05:30 [Info] ----------------------------------------
2019-12-10T22:37:35.041+05:30 [Info] 
2019-12-10T22:37:35.041+05:30 [Info] Indexer serverGroup:Group 1, nodeId:127.0.0.1:9001, nodeUUID:280ac5e1026b443deea667c0a12beceb, useLiveData:true
2019-12-10T22:37:35.041+05:30 [Info] Indexer total memory:1258291200 (1.17188G), mem:1258291200 (1.17188G), overhead:0 (0), min:1261539620 (1.1749G), data:1274533309 (1.187G) cpu:0.2136, io:152228 (148.66K), scan:258 drain:246 numIndexes:6
2019-12-10T22:37:35.041+05:30 [Info] Indexer isDeleted:false isNew:false exclude: meetConstraint:true
2019-12-10T22:37:35.041+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:37:35.041+05:30 [Info] 		Index name:addressidx 0, bucket:default, defnId:9453058082312876815, instId:18069853558395000838, Partition: 0, new/moved:false
2019-12-10T22:37:35.041+05:30 [Info] 		Index total memory:0 (0), mem:0 (0), overhead:0 (0), min:1009522 (985.861K), data:5047611 (4.81378M) cpu:0.2136 io:152228 (148.66K) scan:258 drain:246
2019-12-10T22:37:35.041+05:30 [Info] 		Index resident:0% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:37:35.041+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:37:35.041+05:30 [Info] 		Index name:index_companyname 0, bucket:default, defnId:13362352446024974117, instId:8926027219439387868, Partition: 0, new/moved:false
2019-12-10T22:37:35.042+05:30 [Info] 		Index total memory:0 (0), mem:0 (0), overhead:0 (0), min:681844 (665.863K), data:3409224 (3.25129M) cpu:0.0000 io:0 (0) scan:0 drain:0
2019-12-10T22:37:35.042+05:30 [Info] 		Index resident:0% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:37:35.042+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:37:35.042+05:30 [Info] 		Index name:index_company 0, bucket:default, defnId:6560167570622610970, instId:14131000415694366248, Partition: 0, new/moved:false
2019-12-10T22:37:35.042+05:30 [Info] 		Index total memory:0 (0), mem:0 (0), overhead:0 (0), min:618052 (603.566K), data:3090260 (2.9471M) cpu:0.0000 io:0 (0) scan:0 drain:0
2019-12-10T22:37:35.042+05:30 [Info] 		Index resident:0% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:37:35.042+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:37:35.042+05:30 [Info] 		Index name:index_company_name_age 0, bucket:default, defnId:16166739134347805642, instId:13682299663548290129, Partition: 0, new/moved:false
2019-12-10T22:37:35.042+05:30 [Info] 		Index total memory:0 (0), mem:0 (0), overhead:0 (0), min:714078 (697.342K), data:3570394 (3.40499M) cpu:0.0000 io:0 (0) scan:0 drain:0
2019-12-10T22:37:35.042+05:30 [Info] 		Index resident:0% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:37:35.042+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:37:35.042+05:30 [Info] 		Index name:index_primary 0, bucket:default, defnId:18131217113229780221, instId:7918048547136816525, Partition: 0, new/moved:false
2019-12-10T22:37:35.042+05:30 [Info] 		Index total memory:0 (0), mem:0 (0), overhead:0 (0), min:224924 (219.652K), data:1124620 (1.07252M) cpu:0.0000 io:0 (0) scan:0 drain:0
2019-12-10T22:37:35.042+05:30 [Info] 		Index resident:0% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:37:35.042+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:37:35.042+05:30 [Info] 		Index name:index_company_name_age_address 0, bucket:default, defnId:10176283198205961312, instId:3261938156081176751, Partition: 0, new/moved:true
2019-12-10T22:37:35.042+05:30 [Info] 		Index total memory:1258291200 (1.17188G), mem:1258291200 (1.17188G), overhead:0 (0), min:1258291200 (1.17188G), data:1258291200 (1.17188G) cpu:0.0000 io:0 (0) scan:0 drain:0
2019-12-10T22:37:35.042+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:37:35.042+05:30 [Info] ****************************************
2019-12-10T22:37:35.042+05:30 [Info] send commit create request to watcher 127.0.0.1:9106
2019-12-10T22:37:35.181+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:35.181+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:35.184+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:35.184+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:35.193+05:30 [Info] switched currmeta from 1391 -> 1393 force true 
2019-12-10T22:37:35.194+05:30 [Info] switched currmeta from 1387 -> 1389 force true 
2019-12-10T22:37:35.318+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:35.318+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:35.323+05:30 [Info] switched currmeta from 1389 -> 1390 force true 
2019-12-10T22:37:35.325+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:35.325+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:35.335+05:30 [Info] switched currmeta from 1393 -> 1394 force true 
2019-12-10T22:37:35.399+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:35.399+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:35.403+05:30 [Info] switched currmeta from 1390 -> 1391 force true 
2019-12-10T22:37:35.408+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:35.408+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:35.412+05:30 [Info] switched currmeta from 1394 -> 1395 force true 
2019-12-10T22:37:35.603+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:35.603+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:35.612+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:35.612+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:35.621+05:30 [Info] switched currmeta from 1395 -> 1396 force true 
2019-12-10T22:37:35.622+05:30 [Info] switched currmeta from 1391 -> 1392 force true 
2019-12-10T22:37:35.817+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:37:35.873+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:35.873+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:35.880+05:30 [Info] switched currmeta from 1392 -> 1392 force true 
2019-12-10T22:37:35.883+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:35.883+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:35.889+05:30 [Info] switched currmeta from 1396 -> 1396 force true 
2019-12-10T22:37:37.649+05:30 [Info] Rollback time has changed for index inst 7918048547136816525. New rollback time 1575997571077922096
2019-12-10T22:37:37.650+05:30 [Info] Rollback time has changed for index inst 7918048547136816525. New rollback time 1575997571077922096
2019-12-10T22:37:38.006+05:30 [Info] connected with 1 indexers
2019-12-10T22:37:38.006+05:30 [Info] index 9453058082312876815 has replicas [18069853558395000838]: 
2019-12-10T22:37:38.006+05:30 [Info] index 6560167570622610970 has replicas [14131000415694366248]: 
2019-12-10T22:37:38.006+05:30 [Info] index 18131217113229780221 has replicas [7918048547136816525]: 
2019-12-10T22:37:38.006+05:30 [Info] index 13362352446024974117 has replicas [8926027219439387868]: 
2019-12-10T22:37:38.006+05:30 [Info] index 16166739134347805642 has replicas [13682299663548290129]: 
2019-12-10T22:37:38.006+05:30 [Info] index 10176283198205961312 has replicas [15833575989994718739]: 
2019-12-10T22:37:38.006+05:30 [Info] index 16166739134347805642 has equivalents [16166739134347805642]: 
2019-12-10T22:37:38.006+05:30 [Info] index 10176283198205961312 has equivalents [10176283198205961312]: 
2019-12-10T22:37:38.006+05:30 [Info] index 9453058082312876815 has equivalents [9453058082312876815]: 
2019-12-10T22:37:38.007+05:30 [Info] index 6560167570622610970 has equivalents [6560167570622610970]: 
2019-12-10T22:37:38.007+05:30 [Info] index 18131217113229780221 has equivalents [18131217113229780221]: 
2019-12-10T22:37:38.007+05:30 [Info] index 13362352446024974117 has equivalents [13362352446024974117]: 
2019-12-10T22:37:38.007+05:30 [Info] client load stats {"14131000415694366248": 0,"7918048547136816525": 0,"8926027219439387868": 0,"13682299663548290129": 0,"15833575989994718739": 0,"18069853558395000838": 5.29090964484375e+08}
2019-12-10T22:37:38.007+05:30 [Info] client hit stats {"13682299663548290129": 0,"15833575989994718739": 0,"18069853558395000838": 7,"14131000415694366248": 0,"7918048547136816525": 0,"8926027219439387868": 0}
2019-12-10T22:37:38.007+05:30 [Info] client pending item stats {"18069853558395000838": 9223372036854775807,"14131000415694366248": 9223372036854775807,"7918048547136816525": 9223372036854775807,"8926027219439387868": 9223372036854775807,"13682299663548290129": 9223372036854775807,"15833575989994718739": 9223372036854775807}
2019-12-10T22:37:38.007+05:30 [Info] client stats current {"14131000415694366248": true,"7918048547136816525": true,"8926027219439387868": true,"13682299663548290129": true,"15833575989994718739": true,"18069853558395000838": true}
2019-12-10T22:37:38.012+05:30 [Info] num concurrent scans {0}
2019-12-10T22:37:38.012+05:30 [Info] average scan response {3042 ms}
2019-12-10T22:37:39.455+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:39.455+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:39.459+05:30 [Info] switched currmeta from 1396 -> 1397 force true 
2019-12-10T22:37:39.460+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:39.460+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:39.465+05:30 [Info] switched currmeta from 1392 -> 1393 force true 
2019-12-10T22:37:40.062+05:30 [Info] CreateIndex 10176283198205961312 default/index_company_name_age_address using:forestdb exprType:N1QL whereExpr:() secExprs:([`company` `name` `age` `address`]) desc:[] isPrimary:false scheme:SINGLE  partitionKeys:([]) with: - elapsed(5.627285832s) err()
2019/12/10 22:37:40 Created the secondary index index_company_name_age_address. Waiting for it become active
2019-12-10T22:37:40.062+05:30 [Info] metadata provider version changed 1397 -> 1398
2019-12-10T22:37:40.062+05:30 [Info] switched currmeta from 1397 -> 1398 force false 
2019/12/10 22:37:40 Index is now active
2019-12-10T22:37:40.063+05:30 [Info] send prepare create request to watcher 127.0.0.1:9106
2019-12-10T22:37:40.067+05:30 [Info] Indexer 127.0.0.1:9106 accept prepare request. Index (default, index_company_name_age_address_friends)
2019-12-10T22:37:40.154+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:40.154+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:40.154+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:40.154+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:40.157+05:30 [Info] switched currmeta from 1398 -> 1398 force true 
2019-12-10T22:37:40.158+05:30 [Info] switched currmeta from 1393 -> 1394 force true 
2019-12-10T22:37:40.241+05:30 [Info] Planner::processCreateToken: There is no create token to process for node 127.0.0.1:9001
2019-12-10T22:37:40.585+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node 127.0.0.1:9001
2019-12-10T22:37:40.700+05:30 [Info] ************ Indexer Layout *************
2019-12-10T22:37:40.700+05:30 [Info] Score: 0
2019-12-10T22:37:40.700+05:30 [Info] variation: 0
2019-12-10T22:37:40.700+05:30 [Info] ElapsedTime: 14ms
2019-12-10T22:37:40.700+05:30 [Info] ConvergenceTime: 2us
2019-12-10T22:37:40.700+05:30 [Info] Iteration: 0
2019-12-10T22:37:40.700+05:30 [Info] Move: 0
2019-12-10T22:37:40.700+05:30 [Info] ----------------------------------------
2019-12-10T22:37:40.700+05:30 [Info] Indexer Memory Mean 1258291200 (1.17188G)
2019-12-10T22:37:40.700+05:30 [Info] Indexer Memory Deviation 0 (0) (0.00%)
2019-12-10T22:37:40.700+05:30 [Info] Indexer Memory Utilization 0.8000
2019-12-10T22:37:40.700+05:30 [Info] Indexer CPU Mean 0.2136
2019-12-10T22:37:40.700+05:30 [Info] Indexer CPU Deviation 0.00 (0.00%)
2019-12-10T22:37:40.700+05:30 [Info] Indexer CPU Utilization 0.0534
2019-12-10T22:37:40.700+05:30 [Info] Indexer IO Mean 152228.0000
2019-12-10T22:37:40.700+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2019-12-10T22:37:40.700+05:30 [Info] Indexer Drain Rate Mean 246.0000
2019-12-10T22:37:40.700+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2019-12-10T22:37:40.700+05:30 [Info] Indexer Scan Rate Mean 258.0000
2019-12-10T22:37:40.700+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2019-12-10T22:37:40.700+05:30 [Info] Indexer Data Size Mean 1280164487 (1.19225G)
2019-12-10T22:37:40.700+05:30 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2019-12-10T22:37:40.700+05:30 [Info] Total Index Data (from non-deleted node) 0
2019-12-10T22:37:40.700+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2019-12-10T22:37:40.700+05:30 [Info] No. Index (from non-deleted node) 0
2019-12-10T22:37:40.700+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2019-12-10T22:37:40.700+05:30 [Info] ----------------------------------------
2019-12-10T22:37:40.700+05:30 [Info] Number of indexes: 7
2019-12-10T22:37:40.700+05:30 [Info] Number of indexers: 1
2019-12-10T22:37:40.700+05:30 [Info] Avg Index Size: 179755885 (171.429M)
2019-12-10T22:37:40.700+05:30 [Info] Max Index Size: 1258291200 (1.17188G)
2019-12-10T22:37:40.700+05:30 [Info] Max Indexer Overhead: 0 (0)
2019-12-10T22:37:40.700+05:30 [Info] Avg Index Cpu: 0.0305
2019-12-10T22:37:40.700+05:30 [Info] Max Index Cpu: 0.2136
2019-12-10T22:37:40.700+05:30 [Info] Num Estimation Run: 1
2019-12-10T22:37:40.700+05:30 [Info] ----------------------------------------
2019-12-10T22:37:40.700+05:30 [Info] Memory Quota 1572864000 (1.46484G)
2019-12-10T22:37:40.700+05:30 [Info] CPU Quota 4
2019-12-10T22:37:40.700+05:30 [Info] Max Cpu Utilization -1
2019-12-10T22:37:40.700+05:30 [Info] Max Memory Utilization -1
2019-12-10T22:37:40.700+05:30 [Info] ----------------------------------------
2019-12-10T22:37:40.700+05:30 [Info] Total iteration: 0
2019-12-10T22:37:40.700+05:30 [Info] RandomSwap time: 0ns
2019-12-10T22:37:40.700+05:30 [Info] RandomSwap call: 0
2019-12-10T22:37:40.700+05:30 [Info] RandomSwap iteration: 0
2019-12-10T22:37:40.700+05:30 [Info] RandomMove time: 0ns
2019-12-10T22:37:40.701+05:30 [Info] RandomMove call: 0 (empty index 0)
2019-12-10T22:37:40.701+05:30 [Info] ExhaustMove time: 0ns
2019-12-10T22:37:40.701+05:30 [Info] ExhaustMove call: 0
2019-12-10T22:37:40.701+05:30 [Info] ExhaustSwap time: 0ns
2019-12-10T22:37:40.701+05:30 [Info] ExhaustSwap call: 0
2019-12-10T22:37:40.701+05:30 [Info] ----------------------------------------
2019-12-10T22:37:40.701+05:30 [Info] 
2019-12-10T22:37:40.701+05:30 [Info] Indexer serverGroup:Group 1, nodeId:127.0.0.1:9001, nodeUUID:280ac5e1026b443deea667c0a12beceb, useLiveData:true
2019-12-10T22:37:40.701+05:30 [Info] Indexer total memory:1258291200 (1.17188G), mem:1258291200 (1.17188G), overhead:0 (0), min:1262665855 (1.17595G), data:1280164487 (1.19225G) cpu:0.2136, io:152228 (148.66K), scan:258 drain:246 numIndexes:7
2019-12-10T22:37:40.701+05:30 [Info] Indexer isDeleted:false isNew:false exclude: meetConstraint:true
2019-12-10T22:37:40.701+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:37:40.701+05:30 [Info] 		Index name:index_company_name_age_address 0, bucket:default, defnId:10176283198205961312, instId:15833575989994718739, Partition: 0, new/moved:false
2019-12-10T22:37:40.701+05:30 [Info] 		Index total memory:0 (0), mem:0 (0), overhead:0 (0), min:1126235 (1.07406M), data:5631178 (5.37031M) cpu:0.0000 io:0 (0) scan:0 drain:0
2019-12-10T22:37:40.701+05:30 [Info] 		Index resident:0% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:37:40.701+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:37:40.701+05:30 [Info] 		Index name:addressidx 0, bucket:default, defnId:9453058082312876815, instId:18069853558395000838, Partition: 0, new/moved:false
2019-12-10T22:37:40.701+05:30 [Info] 		Index total memory:0 (0), mem:0 (0), overhead:0 (0), min:1009522 (985.861K), data:5047611 (4.81378M) cpu:0.2136 io:152228 (148.66K) scan:258 drain:246
2019-12-10T22:37:40.701+05:30 [Info] 		Index resident:0% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:37:40.701+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:37:40.701+05:30 [Info] 		Index name:index_companyname 0, bucket:default, defnId:13362352446024974117, instId:8926027219439387868, Partition: 0, new/moved:false
2019-12-10T22:37:40.701+05:30 [Info] 		Index total memory:0 (0), mem:0 (0), overhead:0 (0), min:681844 (665.863K), data:3409224 (3.25129M) cpu:0.0000 io:0 (0) scan:0 drain:0
2019-12-10T22:37:40.701+05:30 [Info] 		Index resident:0% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:37:40.701+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:37:40.701+05:30 [Info] 		Index name:index_company 0, bucket:default, defnId:6560167570622610970, instId:14131000415694366248, Partition: 0, new/moved:false
2019-12-10T22:37:40.701+05:30 [Info] 		Index total memory:0 (0), mem:0 (0), overhead:0 (0), min:618052 (603.566K), data:3090260 (2.9471M) cpu:0.0000 io:0 (0) scan:0 drain:0
2019-12-10T22:37:40.701+05:30 [Info] 		Index resident:0% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:37:40.701+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:37:40.701+05:30 [Info] 		Index name:index_company_name_age 0, bucket:default, defnId:16166739134347805642, instId:13682299663548290129, Partition: 0, new/moved:false
2019-12-10T22:37:40.701+05:30 [Info] 		Index total memory:0 (0), mem:0 (0), overhead:0 (0), min:714078 (697.342K), data:3570394 (3.40499M) cpu:0.0000 io:0 (0) scan:0 drain:0
2019-12-10T22:37:40.701+05:30 [Info] 		Index resident:0% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:37:40.701+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:37:40.701+05:30 [Info] 		Index name:index_primary 0, bucket:default, defnId:18131217113229780221, instId:7918048547136816525, Partition: 0, new/moved:false
2019-12-10T22:37:40.701+05:30 [Info] 		Index total memory:0 (0), mem:0 (0), overhead:0 (0), min:224924 (219.652K), data:1124620 (1.07252M) cpu:0.0000 io:0 (0) scan:0 drain:0
2019-12-10T22:37:40.701+05:30 [Info] 		Index resident:0% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:37:40.701+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:37:40.701+05:30 [Info] 		Index name:index_company_name_age_address_friends 0, bucket:default, defnId:16372997812767057588, instId:11450045849750070772, Partition: 0, new/moved:true
2019-12-10T22:37:40.701+05:30 [Info] 		Index total memory:1258291200 (1.17188G), mem:1258291200 (1.17188G), overhead:0 (0), min:1258291200 (1.17188G), data:1258291200 (1.17188G) cpu:0.0000 io:0 (0) scan:0 drain:0
2019-12-10T22:37:40.701+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:37:40.701+05:30 [Info] ****************************************
2019-12-10T22:37:40.701+05:30 [Info] send commit create request to watcher 127.0.0.1:9106
2019-12-10T22:37:40.819+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] active conns 0, free conns 2
2019-12-10T22:37:40.825+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:37:40.849+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:40.849+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:40.852+05:30 [Info] switched currmeta from 1398 -> 1400 force true 
2019-12-10T22:37:40.855+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:40.855+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:40.862+05:30 [Info] switched currmeta from 1394 -> 1396 force true 
2019-12-10T22:37:40.898+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:40.898+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:40.902+05:30 [Info] switched currmeta from 1400 -> 1400 force true 
2019-12-10T22:37:40.906+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:40.906+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:40.910+05:30 [Info] switched currmeta from 1396 -> 1396 force true 
2019-12-10T22:37:41.024+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:41.024+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:41.029+05:30 [Info] switched currmeta from 1396 -> 1397 force true 
2019-12-10T22:37:41.033+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:41.033+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:41.037+05:30 [Info] switched currmeta from 1400 -> 1401 force true 
2019-12-10T22:37:41.078+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:41.078+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:41.085+05:30 [Info] switched currmeta from 1397 -> 1398 force true 
2019-12-10T22:37:41.089+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:41.089+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:41.095+05:30 [Info] switched currmeta from 1401 -> 1402 force true 
2019-12-10T22:37:41.258+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:41.258+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:41.261+05:30 [Info] switched currmeta from 1402 -> 1403 force true 
2019-12-10T22:37:41.266+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:41.266+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:41.270+05:30 [Info] switched currmeta from 1398 -> 1399 force true 
2019-12-10T22:37:42.685+05:30 [Info] Rollback time has changed for index inst 15833575989994718739. New rollback time 1575997571077922096
2019-12-10T22:37:42.685+05:30 [Info] Rollback time has changed for index inst 15833575989994718739. New rollback time 1575997571077922096
2019-12-10T22:37:45.194+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:45.194+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:45.195+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:45.195+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:45.197+05:30 [Info] switched currmeta from 1399 -> 1400 force true 
2019-12-10T22:37:45.197+05:30 [Info] switched currmeta from 1403 -> 1404 force true 
2019-12-10T22:37:45.822+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:37:45.867+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:45.867+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:45.869+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:45.869+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:45.874+05:30 [Info] switched currmeta from 1404 -> 1404 force true 
2019-12-10T22:37:45.875+05:30 [Info] switched currmeta from 1400 -> 1400 force true 
2019-12-10T22:37:45.892+05:30 [Info] CreateIndex 16372997812767057588 default/index_company_name_age_address_friends using:forestdb exprType:N1QL whereExpr:() secExprs:([`company` `name` `age` `address` `friends`]) desc:[] isPrimary:false scheme:SINGLE  partitionKeys:([]) with: - elapsed(5.829102163s) err()
2019/12/10 22:37:45 Created the secondary index index_company_name_age_address_friends. Waiting for it become active
2019-12-10T22:37:45.892+05:30 [Info] metadata provider version changed 1404 -> 1405
2019-12-10T22:37:45.892+05:30 [Info] switched currmeta from 1404 -> 1405 force false 
2019/12/10 22:37:45 Index is now active
--- PASS: TestMultiScanSetup (118.71s)
=== RUN   TestMultiScanCount
2019/12/10 22:37:45 In TestMultiScanCount()
2019/12/10 22:37:45 

--------- Composite Index with 2 fields ---------
2019/12/10 22:37:45 
--- ScanAllNoFilter ---
2019/12/10 22:37:45 distinct = false
2019-12-10T22:37:46.023+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:46.023+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:46.024+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:37:46.024+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:37:46.032+05:30 [Info] switched currmeta from 1405 -> 1405 force true 
2019-12-10T22:37:46.032+05:30 [Info] switched currmeta from 1400 -> 1401 force true 
2019/12/10 22:37:46 Using n1ql client
2019-12-10T22:37:46.547+05:30 [Info] GsiClient::UpdateUsecjson: using collatejson as data format between indexer and GsiClient
2019/12/10 22:37:46 MultiScanCount = 10002 ExpectedMultiScanCount = 10002
2019/12/10 22:37:46 
--- ScanAllFilterNil ---
2019/12/10 22:37:46 distinct = false
2019/12/10 22:37:47 Using n1ql client
2019/12/10 22:37:47 MultiScanCount = 10002 ExpectedMultiScanCount = 10002
2019/12/10 22:37:47 
--- ScanAll_AllFiltersNil ---
2019/12/10 22:37:47 distinct = false
2019/12/10 22:37:47 Using n1ql client
2019/12/10 22:37:47 MultiScanCount = 10002 ExpectedMultiScanCount = 10002
2019/12/10 22:37:47 
--- SingleSeek ---
2019/12/10 22:37:47 distinct = false
2019/12/10 22:37:48 Using n1ql client
2019/12/10 22:37:48 MultiScanCount = 1 ExpectedMultiScanCount = 1
2019/12/10 22:37:48 
--- MultipleSeek ---
2019/12/10 22:37:48 distinct = false
2019/12/10 22:37:48 Using n1ql client
2019/12/10 22:37:48 MultiScanCount = 2 ExpectedMultiScanCount = 2
2019/12/10 22:37:48 
--- SimpleRange ---
2019/12/10 22:37:48 distinct = false
2019/12/10 22:37:49 Using n1ql client
2019/12/10 22:37:49 MultiScanCount = 2273 ExpectedMultiScanCount = 2273
2019/12/10 22:37:49 
--- NonOverlappingRanges ---
2019/12/10 22:37:49 distinct = false
2019/12/10 22:37:49 Using n1ql client
2019/12/10 22:37:49 MultiScanCount = 4283 ExpectedMultiScanCount = 4283
2019/12/10 22:37:49 
--- OverlappingRanges ---
2019/12/10 22:37:49 distinct = false
2019/12/10 22:37:50 Using n1ql client
2019/12/10 22:37:50 MultiScanCount = 5756 ExpectedMultiScanCount = 5756
2019/12/10 22:37:50 
--- NonOverlappingFilters ---
2019/12/10 22:37:50 distinct = false
2019/12/10 22:37:50 Using n1ql client
2019/12/10 22:37:50 MultiScanCount = 337 ExpectedMultiScanCount = 337
2019/12/10 22:37:50 
--- OverlappingFilters ---
2019/12/10 22:37:50 distinct = false
2019/12/10 22:37:51 Using n1ql client
2019/12/10 22:37:51 MultiScanCount = 2559 ExpectedMultiScanCount = 2559
2019/12/10 22:37:51 
--- BoundaryFilters ---
2019/12/10 22:37:51 distinct = false
2019/12/10 22:37:52 Using n1ql client
2019/12/10 22:37:52 MultiScanCount = 499 ExpectedMultiScanCount = 499
2019/12/10 22:37:52 
--- SeekAndFilters_NonOverlapping ---
2019/12/10 22:37:52 distinct = false
2019/12/10 22:37:52 Using n1ql client
2019/12/10 22:37:52 MultiScanCount = 256 ExpectedMultiScanCount = 256
2019/12/10 22:37:52 
--- SeekAndFilters_Overlapping ---
2019/12/10 22:37:52 distinct = false
2019/12/10 22:37:52 Using n1ql client
2019/12/10 22:37:53 MultiScanCount = 255 ExpectedMultiScanCount = 255
2019/12/10 22:37:53 
--- SimpleRangeLowUnbounded ---
2019/12/10 22:37:53 distinct = false
2019/12/10 22:37:53 Using n1ql client
2019/12/10 22:37:53 MultiScanCount = 5618 ExpectedMultiScanCount = 5618
2019/12/10 22:37:53 
--- SimpleRangeHighUnbounded ---
2019/12/10 22:37:53 distinct = false
2019/12/10 22:37:53 Using n1ql client
2019/12/10 22:37:54 MultiScanCount = 3704 ExpectedMultiScanCount = 3704
2019/12/10 22:37:54 
--- SimpleRangeMultipleUnbounded ---
2019/12/10 22:37:54 distinct = false
2019/12/10 22:37:54 Using n1ql client
2019/12/10 22:37:54 MultiScanCount = 10002 ExpectedMultiScanCount = 10002
2019/12/10 22:37:54 
--- FiltersWithUnbounded ---
2019/12/10 22:37:54 distinct = false
2019/12/10 22:37:55 Using n1ql client
2019/12/10 22:37:55 MultiScanCount = 3173 ExpectedMultiScanCount = 3173
2019/12/10 22:37:55 
--- FiltersLowGreaterThanHigh ---
2019/12/10 22:37:55 distinct = false
2019/12/10 22:37:55 Using n1ql client
2019/12/10 22:37:55 MultiScanCount = 418 ExpectedMultiScanCount = 418
2019/12/10 22:37:55 

--------- Simple Index with 1 field ---------
2019/12/10 22:37:55 
--- SingleIndexSimpleRange ---
2019/12/10 22:37:55 distinct = false
2019/12/10 22:37:56 Using n1ql client
2019/12/10 22:37:56 MultiScanCount = 2273 ExpectedMultiScanCount = 2273
2019/12/10 22:37:56 
--- SingleIndex_SimpleRanges_NonOverlapping ---
2019/12/10 22:37:56 distinct = false
2019/12/10 22:37:56 Using n1ql client
2019/12/10 22:37:56 MultiScanCount = 7140 ExpectedMultiScanCount = 7140
2019/12/10 22:37:56 
--- SingleIndex_SimpleRanges_Overlapping ---
2019/12/10 22:37:56 distinct = false
2019/12/10 22:37:57 Using n1ql client
2019/12/10 22:37:57 MultiScanCount = 8701 ExpectedMultiScanCount = 8701
2019/12/10 22:37:57 

--------- Composite Index with 3 fields ---------
2019/12/10 22:37:57 
--- ScanAllNoFilter ---
2019/12/10 22:37:57 distinct = false
2019/12/10 22:37:57 Using n1ql client
2019/12/10 22:37:57 MultiScanCount = 10002 ExpectedMultiScanCount = 10002
2019/12/10 22:37:57 
--- ScanAllFilterNil ---
2019/12/10 22:37:57 distinct = false
2019/12/10 22:37:58 Using n1ql client
2019/12/10 22:37:58 MultiScanCount = 10002 ExpectedMultiScanCount = 10002
2019/12/10 22:37:58 
--- ScanAll_AllFiltersNil ---
2019/12/10 22:37:58 distinct = false
2019/12/10 22:37:59 Using n1ql client
2019/12/10 22:37:59 MultiScanCount = 10002 ExpectedMultiScanCount = 10002
2019/12/10 22:37:59 
--- 3FieldsSingleSeek ---
2019/12/10 22:37:59 distinct = false
2019/12/10 22:37:59 Using n1ql client
2019/12/10 22:37:59 MultiScanCount = 1 ExpectedMultiScanCount = 1
2019/12/10 22:37:59 
--- 3FieldsMultipleSeeks ---
2019/12/10 22:37:59 distinct = false
2019/12/10 22:38:00 Using n1ql client
2019/12/10 22:38:00 MultiScanCount = 3 ExpectedMultiScanCount = 3
2019/12/10 22:38:00 
--- 3FieldsMultipleSeeks_Identical ---
2019/12/10 22:38:00 distinct = false
2019/12/10 22:38:00 Using n1ql client
2019/12/10 22:38:00 MultiScanCount = 2 ExpectedMultiScanCount = 2
2019/12/10 22:38:00 

--------- New scenarios ---------
2019/12/10 22:38:00 
--- CompIndexHighUnbounded1 ---
2019/12/10 22:38:00 
--- Multi Scan 0 ---
2019/12/10 22:38:00 distinct = false
2019/12/10 22:38:01 Using n1ql client
2019/12/10 22:38:01 Using n1ql client
2019/12/10 22:38:01 len(scanResults) = 8 MultiScanCount = 8
2019/12/10 22:38:01 Expected and Actual scan responses are the same
2019/12/10 22:38:01 
--- Multi Scan 1 ---
2019/12/10 22:38:01 distinct = false
2019/12/10 22:38:01 Using n1ql client
2019/12/10 22:38:01 Using n1ql client
2019/12/10 22:38:01 len(scanResults) = 0 MultiScanCount = 0
2019/12/10 22:38:01 Expected and Actual scan responses are the same
2019/12/10 22:38:01 
--- Multi Scan 2 ---
2019/12/10 22:38:01 distinct = false
2019/12/10 22:38:02 Using n1ql client
2019/12/10 22:38:02 Using n1ql client
2019/12/10 22:38:02 len(scanResults) = 9 MultiScanCount = 9
2019/12/10 22:38:02 Expected and Actual scan responses are the same
2019/12/10 22:38:02 
--- CompIndexHighUnbounded2 ---
2019/12/10 22:38:02 
--- Multi Scan 0 ---
2019/12/10 22:38:02 distinct = false
2019/12/10 22:38:02 Using n1ql client
2019/12/10 22:38:02 Using n1ql client
2019/12/10 22:38:02 len(scanResults) = 4138 MultiScanCount = 4138
2019/12/10 22:38:02 Expected and Actual scan responses are the same
2019/12/10 22:38:02 
--- Multi Scan 1 ---
2019/12/10 22:38:02 distinct = false
2019/12/10 22:38:03 Using n1ql client
2019/12/10 22:38:03 Using n1ql client
2019/12/10 22:38:03 len(scanResults) = 2746 MultiScanCount = 2746
2019/12/10 22:38:03 Expected and Actual scan responses are the same
2019/12/10 22:38:03 
--- Multi Scan 2 ---
2019/12/10 22:38:03 distinct = false
2019/12/10 22:38:04 Using n1ql client
2019/12/10 22:38:04 Using n1ql client
2019/12/10 22:38:04 len(scanResults) = 4691 MultiScanCount = 4691
2019/12/10 22:38:04 Expected and Actual scan responses are the same
2019/12/10 22:38:04 
--- CompIndexHighUnbounded3 ---
2019/12/10 22:38:04 
--- Multi Scan 0 ---
2019/12/10 22:38:04 distinct = false
2019/12/10 22:38:04 Using n1ql client
2019/12/10 22:38:04 Using n1ql client
2019/12/10 22:38:04 len(scanResults) = 1329 MultiScanCount = 1329
2019/12/10 22:38:04 Expected and Actual scan responses are the same
2019/12/10 22:38:04 
--- CompIndexHighUnbounded4 ---
2019/12/10 22:38:04 
--- Multi Scan 0 ---
2019/12/10 22:38:04 distinct = false
2019/12/10 22:38:05 Using n1ql client
2019/12/10 22:38:05 Using n1ql client
2019/12/10 22:38:05 len(scanResults) = 5349 MultiScanCount = 5349
2019/12/10 22:38:05 Expected and Actual scan responses are the same
2019/12/10 22:38:05 
--- CompIndexHighUnbounded5 ---
2019/12/10 22:38:05 
--- Multi Scan 0 ---
2019/12/10 22:38:05 distinct = false
2019/12/10 22:38:06 Using n1ql client
2019/12/10 22:38:06 Using n1ql client
2019/12/10 22:38:06 len(scanResults) = 8210 MultiScanCount = 8210
2019/12/10 22:38:06 Expected and Actual scan responses are the same
2019/12/10 22:38:06 
--- SeekBoundaries ---
2019/12/10 22:38:06 
--- Multi Scan 0 ---
2019/12/10 22:38:06 distinct = false
2019/12/10 22:38:06 Using n1ql client
2019/12/10 22:38:06 Using n1ql client
2019/12/10 22:38:06 len(scanResults) = 175 MultiScanCount = 175
2019/12/10 22:38:06 Expected and Actual scan responses are the same
2019/12/10 22:38:06 
--- Multi Scan 1 ---
2019/12/10 22:38:06 distinct = false
2019/12/10 22:38:07 Using n1ql client
2019/12/10 22:38:07 Using n1ql client
2019/12/10 22:38:07 len(scanResults) = 1 MultiScanCount = 1
2019/12/10 22:38:07 Expected and Actual scan responses are the same
2019/12/10 22:38:07 
--- Multi Scan 2 ---
2019/12/10 22:38:07 distinct = false
2019/12/10 22:38:07 Using n1ql client
2019/12/10 22:38:07 Using n1ql client
2019/12/10 22:38:08 len(scanResults) = 555 MultiScanCount = 555
2019/12/10 22:38:08 Expected and Actual scan responses are the same
2019/12/10 22:38:08 
--- Multi Scan 3 ---
2019/12/10 22:38:08 distinct = false
2019/12/10 22:38:08 Using n1ql client
2019/12/10 22:38:08 Using n1ql client
2019/12/10 22:38:08 len(scanResults) = 872 MultiScanCount = 872
2019/12/10 22:38:08 Expected and Actual scan responses are the same
2019/12/10 22:38:08 
--- Multi Scan 4 ---
2019/12/10 22:38:08 distinct = false
2019/12/10 22:38:09 Using n1ql client
2019/12/10 22:38:09 Using n1ql client
2019/12/10 22:38:09 len(scanResults) = 287 MultiScanCount = 287
2019/12/10 22:38:09 Expected and Actual scan responses are the same
2019/12/10 22:38:09 
--- Multi Scan 5 ---
2019/12/10 22:38:09 distinct = false
2019/12/10 22:38:09 Using n1ql client
2019/12/10 22:38:09 Using n1ql client
2019/12/10 22:38:09 len(scanResults) = 5254 MultiScanCount = 5254
2019/12/10 22:38:09 Expected and Actual scan responses are the same
2019/12/10 22:38:09 
--- Multi Scan 6 ---
2019/12/10 22:38:09 distinct = false
2019/12/10 22:38:10 Using n1ql client
2019/12/10 22:38:10 Using n1ql client
2019/12/10 22:38:10 len(scanResults) = 5566 MultiScanCount = 5566
2019/12/10 22:38:10 Expected and Actual scan responses are the same
2019/12/10 22:38:10 
--- Multi Scan 7 ---
2019/12/10 22:38:10 distinct = false
2019/12/10 22:38:11 Using n1ql client
2019/12/10 22:38:11 Using n1ql client
2019/12/10 22:38:11 len(scanResults) = 8 MultiScanCount = 8
2019/12/10 22:38:11 Expected and Actual scan responses are the same
2019/12/10 22:38:11 

--------- With DISTINCT True ---------
2019/12/10 22:38:11 
--- ScanAllNoFilter ---
2019/12/10 22:38:11 distinct = true
2019/12/10 22:38:11 Using n1ql client
2019/12/10 22:38:11 MultiScanCount = 999 ExpectedMultiScanCount = 999
2019/12/10 22:38:11 
--- ScanAllFilterNil ---
2019/12/10 22:38:11 distinct = true
2019/12/10 22:38:12 Using n1ql client
2019/12/10 22:38:12 MultiScanCount = 999 ExpectedMultiScanCount = 999
2019/12/10 22:38:12 
--- ScanAll_AllFiltersNil ---
2019/12/10 22:38:12 distinct = true
2019/12/10 22:38:12 Using n1ql client
2019/12/10 22:38:13 MultiScanCount = 999 ExpectedMultiScanCount = 999
2019/12/10 22:38:13 
--- SingleSeek ---
2019/12/10 22:38:13 distinct = true
2019/12/10 22:38:13 Using n1ql client
2019/12/10 22:38:13 MultiScanCount = 1 ExpectedMultiScanCount = 1
2019/12/10 22:38:13 
--- MultipleSeek ---
2019/12/10 22:38:13 distinct = true
2019/12/10 22:38:13 Using n1ql client
2019/12/10 22:38:13 MultiScanCount = 2 ExpectedMultiScanCount = 2
2019/12/10 22:38:13 
--- SimpleRange ---
2019/12/10 22:38:13 distinct = true
2019/12/10 22:38:14 Using n1ql client
2019/12/10 22:38:14 MultiScanCount = 227 ExpectedMultiScanCount = 227
2019/12/10 22:38:14 
--- NonOverlappingRanges ---
2019/12/10 22:38:14 distinct = true
2019/12/10 22:38:14 Using n1ql client
2019/12/10 22:38:15 MultiScanCount = 428 ExpectedMultiScanCount = 428
2019/12/10 22:38:15 
--- OverlappingRanges ---
2019/12/10 22:38:15 distinct = true
2019/12/10 22:38:15 Using n1ql client
2019/12/10 22:38:15 MultiScanCount = 575 ExpectedMultiScanCount = 575
2019/12/10 22:38:15 
--- NonOverlappingFilters ---
2019/12/10 22:38:15 distinct = true
2019/12/10 22:38:16 Using n1ql client
2019/12/10 22:38:16 MultiScanCount = 186 ExpectedMultiScanCount = 186
2019/12/10 22:38:16 
--- NonOverlappingFilters2 ---
2019/12/10 22:38:16 distinct = true
2019/12/10 22:38:16 Using n1ql client
2019/12/10 22:38:16 MultiScanCount = 1 ExpectedMultiScanCount = 1
2019/12/10 22:38:16 
--- OverlappingFilters ---
2019/12/10 22:38:16 distinct = true
2019/12/10 22:38:17 Using n1ql client
2019/12/10 22:38:17 MultiScanCount = 543 ExpectedMultiScanCount = 543
2019/12/10 22:38:17 
--- BoundaryFilters ---
2019/12/10 22:38:17 distinct = true
2019/12/10 22:38:17 Using n1ql client
2019/12/10 22:38:17 MultiScanCount = 172 ExpectedMultiScanCount = 172
2019/12/10 22:38:17 
--- SeekAndFilters_NonOverlapping ---
2019/12/10 22:38:17 distinct = true
2019/12/10 22:38:18 Using n1ql client
2019/12/10 22:38:18 MultiScanCount = 135 ExpectedMultiScanCount = 135
2019/12/10 22:38:18 
--- SeekAndFilters_Overlapping ---
2019/12/10 22:38:18 distinct = true
2019/12/10 22:38:18 Using n1ql client
2019/12/10 22:38:18 MultiScanCount = 134 ExpectedMultiScanCount = 134
2019/12/10 22:38:18 
--- SimpleRangeLowUnbounded ---
2019/12/10 22:38:18 distinct = false
2019/12/10 22:38:19 Using n1ql client
2019/12/10 22:38:19 MultiScanCount = 5618 ExpectedMultiScanCount = 5618
2019/12/10 22:38:19 
--- SimpleRangeHighUnbounded ---
2019/12/10 22:38:19 distinct = false
2019/12/10 22:38:19 Using n1ql client
2019/12/10 22:38:19 MultiScanCount = 3704 ExpectedMultiScanCount = 3704
2019/12/10 22:38:19 
--- SimpleRangeMultipleUnbounded ---
2019/12/10 22:38:19 distinct = false
2019/12/10 22:38:20 Using n1ql client
2019/12/10 22:38:20 MultiScanCount = 10002 ExpectedMultiScanCount = 10002
2019/12/10 22:38:20 
--- FiltersWithUnbounded ---
2019/12/10 22:38:20 distinct = false
2019/12/10 22:38:20 Using n1ql client
2019/12/10 22:38:20 MultiScanCount = 3173 ExpectedMultiScanCount = 3173
2019/12/10 22:38:20 
--- FiltersLowGreaterThanHigh ---
2019/12/10 22:38:20 distinct = false
2019/12/10 22:38:21 Using n1ql client
2019/12/10 22:38:21 MultiScanCount = 418 ExpectedMultiScanCount = 418
2019/12/10 22:38:21 

--------- Simple Index with 1 field ---------
2019/12/10 22:38:21 
--- SingleIndexSimpleRange ---
2019/12/10 22:38:21 distinct = true
2019/12/10 22:38:21 Using n1ql client
2019/12/10 22:38:21 MultiScanCount = 227 ExpectedMultiScanCount = 227
2019/12/10 22:38:21 
--- SingleIndex_SimpleRanges_NonOverlapping ---
2019/12/10 22:38:21 distinct = true
2019/12/10 22:38:22 Using n1ql client
2019/12/10 22:38:22 MultiScanCount = 713 ExpectedMultiScanCount = 713
2019/12/10 22:38:22 
--- SingleIndex_SimpleRanges_Overlapping ---
2019/12/10 22:38:22 distinct = true
2019/12/10 22:38:22 Using n1ql client
2019/12/10 22:38:22 MultiScanCount = 869 ExpectedMultiScanCount = 869
2019/12/10 22:38:22 

--------- Composite Index with 3 fields ---------
2019/12/10 22:38:22 
--- ScanAllNoFilter ---
2019/12/10 22:38:22 distinct = true
2019/12/10 22:38:23 Using n1ql client
2019/12/10 22:38:23 MultiScanCount = 999 ExpectedMultiScanCount = 999
2019/12/10 22:38:23 
--- ScanAllFilterNil ---
2019/12/10 22:38:23 distinct = true
2019/12/10 22:38:24 Using n1ql client
2019/12/10 22:38:24 MultiScanCount = 999 ExpectedMultiScanCount = 999
2019/12/10 22:38:24 
--- ScanAll_AllFiltersNil ---
2019/12/10 22:38:24 distinct = true
2019/12/10 22:38:24 Using n1ql client
2019/12/10 22:38:25 MultiScanCount = 999 ExpectedMultiScanCount = 999
2019/12/10 22:38:25 
--- 3FieldsSingleSeek ---
2019/12/10 22:38:25 distinct = true
2019/12/10 22:38:25 Using n1ql client
2019/12/10 22:38:25 MultiScanCount = 1 ExpectedMultiScanCount = 1
2019/12/10 22:38:25 
--- 3FieldsMultipleSeeks ---
2019/12/10 22:38:25 distinct = true
2019/12/10 22:38:26 Using n1ql client
2019/12/10 22:38:26 MultiScanCount = 3 ExpectedMultiScanCount = 3
2019/12/10 22:38:26 
--- 3FieldsMultipleSeeks_Identical ---
2019/12/10 22:38:26 distinct = true
2019/12/10 22:38:26 Using n1ql client
2019/12/10 22:38:26 MultiScanCount = 2 ExpectedMultiScanCount = 2
--- PASS: TestMultiScanCount (40.68s)
=== RUN   TestMultiScanScenarios
2019/12/10 22:38:26 In TestMultiScanScenarios()
2019/12/10 22:38:26 

--------- Composite Index with 2 fields ---------
2019/12/10 22:38:26 
--- ScanAllNoFilter ---
2019/12/10 22:38:26 distinct = false
2019/12/10 22:38:27 Using n1ql client
2019/12/10 22:38:27 Expected and Actual scan responses are the same
2019/12/10 22:38:27 
--- ScanAllFilterNil ---
2019/12/10 22:38:27 distinct = false
2019/12/10 22:38:27 Using n1ql client
2019/12/10 22:38:27 Expected and Actual scan responses are the same
2019/12/10 22:38:27 
--- ScanAll_AllFiltersNil ---
2019/12/10 22:38:27 distinct = false
2019/12/10 22:38:28 Using n1ql client
2019/12/10 22:38:28 Expected and Actual scan responses are the same
2019/12/10 22:38:28 
--- SingleSeek ---
2019/12/10 22:38:28 distinct = false
2019/12/10 22:38:29 Using n1ql client
2019/12/10 22:38:29 Expected and Actual scan responses are the same
2019/12/10 22:38:29 
--- MultipleSeek ---
2019/12/10 22:38:29 distinct = false
2019/12/10 22:38:29 Using n1ql client
2019/12/10 22:38:29 Expected and Actual scan responses are the same
2019/12/10 22:38:29 
--- SimpleRange ---
2019/12/10 22:38:29 distinct = false
2019/12/10 22:38:29 Using n1ql client
2019/12/10 22:38:29 Expected and Actual scan responses are the same
2019/12/10 22:38:29 
--- NonOverlappingRanges ---
2019/12/10 22:38:29 distinct = false
2019/12/10 22:38:30 Using n1ql client
2019/12/10 22:38:30 Expected and Actual scan responses are the same
2019/12/10 22:38:30 
--- OverlappingRanges ---
2019/12/10 22:38:30 distinct = false
2019/12/10 22:38:31 Using n1ql client
2019/12/10 22:38:31 Expected and Actual scan responses are the same
2019/12/10 22:38:31 
--- NonOverlappingFilters ---
2019/12/10 22:38:31 distinct = false
2019/12/10 22:38:31 Using n1ql client
2019/12/10 22:38:31 Expected and Actual scan responses are the same
2019/12/10 22:38:31 
--- OverlappingFilters ---
2019/12/10 22:38:31 distinct = false
2019/12/10 22:38:32 Using n1ql client
2019/12/10 22:38:32 Expected and Actual scan responses are the same
2019/12/10 22:38:32 
--- BoundaryFilters ---
2019/12/10 22:38:32 distinct = false
2019/12/10 22:38:32 Using n1ql client
2019/12/10 22:38:32 Expected and Actual scan responses are the same
2019/12/10 22:38:32 
--- SeekAndFilters_NonOverlapping ---
2019/12/10 22:38:32 distinct = false
2019/12/10 22:38:33 Using n1ql client
2019/12/10 22:38:33 Expected and Actual scan responses are the same
2019/12/10 22:38:33 
--- SeekAndFilters_Overlapping ---
2019/12/10 22:38:33 distinct = false
2019/12/10 22:38:33 Using n1ql client
2019/12/10 22:38:33 Expected and Actual scan responses are the same
2019/12/10 22:38:33 
--- SimpleRangeLowUnbounded ---
2019/12/10 22:38:33 distinct = false
2019/12/10 22:38:34 Using n1ql client
2019/12/10 22:38:34 Expected and Actual scan responses are the same
2019/12/10 22:38:34 
--- SimpleRangeHighUnbounded ---
2019/12/10 22:38:34 distinct = false
2019/12/10 22:38:34 Using n1ql client
2019/12/10 22:38:34 Expected and Actual scan responses are the same
2019/12/10 22:38:34 
--- SimpleRangeMultipleUnbounded ---
2019/12/10 22:38:34 distinct = false
2019/12/10 22:38:35 Using n1ql client
2019/12/10 22:38:35 Expected and Actual scan responses are the same
2019/12/10 22:38:35 
--- FiltersWithUnbounded ---
2019/12/10 22:38:35 distinct = false
2019/12/10 22:38:36 Using n1ql client
2019/12/10 22:38:36 Expected and Actual scan responses are the same
2019/12/10 22:38:36 
--- FiltersLowGreaterThanHigh ---
2019/12/10 22:38:36 distinct = false
2019/12/10 22:38:36 Using n1ql client
2019/12/10 22:38:36 Expected and Actual scan responses are the same
2019/12/10 22:38:36 

--------- Simple Index with 1 field ---------
2019/12/10 22:38:36 
--- SingleIndexSimpleRange ---
2019/12/10 22:38:36 distinct = false
2019/12/10 22:38:37 Using n1ql client
2019/12/10 22:38:37 Expected and Actual scan responses are the same
2019/12/10 22:38:37 
--- SingleIndex_SimpleRanges_NonOverlapping ---
2019/12/10 22:38:37 distinct = false
2019/12/10 22:38:37 Using n1ql client
2019/12/10 22:38:37 Expected and Actual scan responses are the same
2019/12/10 22:38:37 
--- SingleIndex_SimpleRanges_Overlapping ---
2019/12/10 22:38:37 distinct = false
2019/12/10 22:38:38 Using n1ql client
2019/12/10 22:38:38 Expected and Actual scan responses are the same
2019/12/10 22:38:38 

--------- Composite Index with 3 fields ---------
2019/12/10 22:38:38 
--- ScanAllNoFilter ---
2019/12/10 22:38:38 distinct = false
2019/12/10 22:38:38 Using n1ql client
2019/12/10 22:38:39 Expected and Actual scan responses are the same
2019/12/10 22:38:39 
--- ScanAllFilterNil ---
2019/12/10 22:38:39 distinct = false
2019/12/10 22:38:39 Using n1ql client
2019/12/10 22:38:39 Expected and Actual scan responses are the same
2019/12/10 22:38:39 
--- ScanAll_AllFiltersNil ---
2019/12/10 22:38:39 distinct = false
2019/12/10 22:38:40 Using n1ql client
2019/12/10 22:38:40 Expected and Actual scan responses are the same
2019/12/10 22:38:40 
--- 3FieldsSingleSeek ---
2019/12/10 22:38:40 distinct = false
2019/12/10 22:38:41 Using n1ql client
2019/12/10 22:38:41 Expected and Actual scan responses are the same
2019/12/10 22:38:41 
--- 3FieldsMultipleSeeks ---
2019/12/10 22:38:41 distinct = false
2019/12/10 22:38:41 Using n1ql client
2019/12/10 22:38:41 Expected and Actual scan responses are the same
2019/12/10 22:38:41 
--- 3FieldsMultipleSeeks_Identical ---
2019/12/10 22:38:41 distinct = false
2019/12/10 22:38:42 Using n1ql client
2019/12/10 22:38:42 Expected and Actual scan responses are the same
2019/12/10 22:38:42 

--------- New scenarios ---------
2019/12/10 22:38:42 
--- CompIndexHighUnbounded1 ---
2019/12/10 22:38:42 
--- Multi Scan 0 ---
2019/12/10 22:38:42 distinct = false
2019/12/10 22:38:42 Using n1ql client
2019/12/10 22:38:42 Expected and Actual scan responses are the same
2019/12/10 22:38:42 
--- Multi Scan 1 ---
2019/12/10 22:38:42 distinct = false
2019/12/10 22:38:42 Using n1ql client
2019/12/10 22:38:42 Expected and Actual scan responses are the same
2019/12/10 22:38:42 
--- Multi Scan 2 ---
2019/12/10 22:38:42 distinct = false
2019/12/10 22:38:43 Using n1ql client
2019/12/10 22:38:43 Expected and Actual scan responses are the same
2019/12/10 22:38:43 
--- CompIndexHighUnbounded2 ---
2019/12/10 22:38:43 
--- Multi Scan 0 ---
2019/12/10 22:38:43 distinct = false
2019/12/10 22:38:43 Using n1ql client
2019/12/10 22:38:44 Expected and Actual scan responses are the same
2019/12/10 22:38:44 
--- Multi Scan 1 ---
2019/12/10 22:38:44 distinct = false
2019/12/10 22:38:44 Using n1ql client
2019/12/10 22:38:44 Expected and Actual scan responses are the same
2019/12/10 22:38:44 
--- Multi Scan 2 ---
2019/12/10 22:38:44 distinct = false
2019/12/10 22:38:45 Using n1ql client
2019/12/10 22:38:45 Expected and Actual scan responses are the same
2019/12/10 22:38:45 
--- CompIndexHighUnbounded3 ---
2019/12/10 22:38:45 
--- Multi Scan 0 ---
2019/12/10 22:38:45 distinct = false
2019/12/10 22:38:45 Using n1ql client
2019/12/10 22:38:45 Expected and Actual scan responses are the same
2019/12/10 22:38:45 
--- CompIndexHighUnbounded4 ---
2019/12/10 22:38:45 
--- Multi Scan 0 ---
2019/12/10 22:38:45 distinct = false
2019/12/10 22:38:46 Using n1ql client
2019/12/10 22:38:46 Expected and Actual scan responses are the same
2019/12/10 22:38:46 
--- CompIndexHighUnbounded5 ---
2019/12/10 22:38:46 
--- Multi Scan 0 ---
2019/12/10 22:38:46 distinct = false
2019/12/10 22:38:46 Using n1ql client
2019/12/10 22:38:47 Expected and Actual scan responses are the same
2019/12/10 22:38:47 
--- SeekBoundaries ---
2019/12/10 22:38:47 
--- Multi Scan 0 ---
2019/12/10 22:38:47 distinct = false
2019/12/10 22:38:47 Using n1ql client
2019/12/10 22:38:47 Expected and Actual scan responses are the same
2019/12/10 22:38:47 
--- Multi Scan 1 ---
2019/12/10 22:38:47 distinct = false
2019/12/10 22:38:47 Using n1ql client
2019/12/10 22:38:47 Expected and Actual scan responses are the same
2019/12/10 22:38:47 
--- Multi Scan 2 ---
2019/12/10 22:38:47 distinct = false
2019/12/10 22:38:48 Using n1ql client
2019/12/10 22:38:48 Expected and Actual scan responses are the same
2019/12/10 22:38:48 
--- Multi Scan 3 ---
2019/12/10 22:38:48 distinct = false
2019/12/10 22:38:48 Using n1ql client
2019/12/10 22:38:49 Expected and Actual scan responses are the same
2019/12/10 22:38:49 
--- Multi Scan 4 ---
2019/12/10 22:38:49 distinct = false
2019/12/10 22:38:49 Using n1ql client
2019/12/10 22:38:49 Expected and Actual scan responses are the same
2019/12/10 22:38:49 
--- Multi Scan 5 ---
2019/12/10 22:38:49 distinct = false
2019/12/10 22:38:49 Using n1ql client
2019/12/10 22:38:50 Expected and Actual scan responses are the same
2019/12/10 22:38:50 
--- Multi Scan 6 ---
2019/12/10 22:38:50 distinct = false
2019/12/10 22:38:50 Using n1ql client
2019/12/10 22:38:50 Expected and Actual scan responses are the same
2019/12/10 22:38:50 
--- Multi Scan 7 ---
2019/12/10 22:38:50 distinct = false
2019/12/10 22:38:51 Using n1ql client
2019/12/10 22:38:51 Expected and Actual scan responses are the same
2019/12/10 22:38:51 
--- PrefixSortVariations ---
2019/12/10 22:38:51 
--- Multi Scan 0 ---
2019/12/10 22:38:51 distinct = false
2019/12/10 22:38:51 Using n1ql client
2019/12/10 22:38:51 Expected and Actual scan responses are the same
2019/12/10 22:38:51 
--- Multi Scan 1 ---
2019/12/10 22:38:51 distinct = false
2019/12/10 22:38:52 Using n1ql client
2019/12/10 22:38:52 Expected and Actual scan responses are the same
--- PASS: TestMultiScanScenarios (25.86s)
=== RUN   TestMultiScanOffset
2019/12/10 22:38:52 In TestMultiScanOffset()
2019/12/10 22:38:52 

--------- Composite Index with 2 fields ---------
2019/12/10 22:38:52 
--- ScanAllNoFilter ---
2019/12/10 22:38:52 distinct = false
2019/12/10 22:38:52 Using n1ql client
2019/12/10 22:38:53 
--- ScanAllFilterNil ---
2019/12/10 22:38:53 distinct = false
2019/12/10 22:38:53 Using n1ql client
2019/12/10 22:38:53 
--- ScanAll_AllFiltersNil ---
2019/12/10 22:38:53 distinct = false
2019/12/10 22:38:54 Using n1ql client
2019/12/10 22:38:54 
--- SingleSeek ---
2019/12/10 22:38:54 distinct = false
2019/12/10 22:38:54 Using n1ql client
2019/12/10 22:38:54 
--- MultipleSeek ---
2019/12/10 22:38:54 distinct = false
2019/12/10 22:38:55 Using n1ql client
2019/12/10 22:38:55 
--- SimpleRange ---
2019/12/10 22:38:55 distinct = false
2019/12/10 22:38:55 Using n1ql client
2019/12/10 22:38:55 
--- NonOverlappingRanges ---
2019/12/10 22:38:55 distinct = false
2019/12/10 22:38:56 Using n1ql client
2019/12/10 22:38:56 
--- OverlappingRanges ---
2019/12/10 22:38:56 distinct = false
2019/12/10 22:38:56 Using n1ql client
2019/12/10 22:38:56 
--- NonOverlappingFilters ---
2019/12/10 22:38:56 distinct = false
2019/12/10 22:38:57 Using n1ql client
2019/12/10 22:38:57 
--- OverlappingFilters ---
2019/12/10 22:38:57 distinct = false
2019/12/10 22:38:57 Using n1ql client
2019/12/10 22:38:57 
--- BoundaryFilters ---
2019/12/10 22:38:57 distinct = false
2019/12/10 22:38:58 Using n1ql client
2019/12/10 22:38:58 
--- SeekAndFilters_NonOverlapping ---
2019/12/10 22:38:58 distinct = false
2019/12/10 22:38:58 Using n1ql client
2019/12/10 22:38:58 
--- SeekAndFilters_Overlapping ---
2019/12/10 22:38:58 distinct = false
2019/12/10 22:38:59 Using n1ql client
2019/12/10 22:38:59 
--- SimpleRangeLowUnbounded ---
2019/12/10 22:38:59 distinct = false
2019/12/10 22:38:59 Using n1ql client
2019/12/10 22:39:00 Expected and Actual scan responses are the same
2019/12/10 22:39:00 
--- SimpleRangeHighUnbounded ---
2019/12/10 22:39:00 distinct = false
2019/12/10 22:39:00 Using n1ql client
2019/12/10 22:39:00 Expected and Actual scan responses are the same
2019/12/10 22:39:00 
--- SimpleRangeMultipleUnbounded ---
2019/12/10 22:39:00 distinct = false
2019/12/10 22:39:00 Using n1ql client
2019/12/10 22:39:01 Expected and Actual scan responses are the same
2019/12/10 22:39:01 
--- FiltersWithUnbounded ---
2019/12/10 22:39:01 distinct = false
2019/12/10 22:39:01 Using n1ql client
2019/12/10 22:39:01 Expected and Actual scan responses are the same
2019/12/10 22:39:01 
--- FiltersLowGreaterThanHigh ---
2019/12/10 22:39:01 distinct = false
2019/12/10 22:39:02 Using n1ql client
2019/12/10 22:39:02 Expected and Actual scan responses are the same
2019/12/10 22:39:02 

--------- Simple Index with 1 field ---------
2019/12/10 22:39:02 
--- SingleIndexSimpleRange ---
2019/12/10 22:39:02 distinct = false
2019/12/10 22:39:02 Using n1ql client
2019/12/10 22:39:02 
--- SingleIndex_SimpleRanges_NonOverlapping ---
2019/12/10 22:39:02 distinct = false
2019/12/10 22:39:03 Using n1ql client
2019/12/10 22:39:03 
--- SingleIndex_SimpleRanges_Overlapping ---
2019/12/10 22:39:03 distinct = false
2019/12/10 22:39:03 Using n1ql client
2019/12/10 22:39:03 

--------- Composite Index with 3 fields ---------
2019/12/10 22:39:03 
--- ScanAllNoFilter ---
2019/12/10 22:39:03 distinct = false
2019/12/10 22:39:04 Using n1ql client
2019/12/10 22:39:04 
--- ScanAllFilterNil ---
2019/12/10 22:39:04 distinct = false
2019/12/10 22:39:04 Using n1ql client
2019/12/10 22:39:05 
--- ScanAll_AllFiltersNil ---
2019/12/10 22:39:05 distinct = false
2019/12/10 22:39:05 Using n1ql client
2019/12/10 22:39:05 
--- 3FieldsSingleSeek ---
2019/12/10 22:39:05 distinct = false
2019/12/10 22:39:06 Using n1ql client
2019/12/10 22:39:06 
--- 3FieldsMultipleSeeks ---
2019/12/10 22:39:06 distinct = false
2019/12/10 22:39:06 Using n1ql client
2019/12/10 22:39:06 
--- 3FieldsMultipleSeeks_Identical ---
2019/12/10 22:39:06 distinct = false
2019/12/10 22:39:07 Using n1ql client
--- PASS: TestMultiScanOffset (15.01s)
=== RUN   TestMultiScanPrimaryIndex
2019/12/10 22:39:07 In TestMultiScanPrimaryIndex()
2019/12/10 22:39:07 
--- PrimaryRange ---
2019/12/10 22:39:07 Using n1ql client
2019/12/10 22:39:07 Expected and Actual scan responses are the same
2019/12/10 22:39:07 
--- PrimaryScanAllNoFilter ---
2019/12/10 22:39:07 Using n1ql client
2019/12/10 22:39:07 Expected and Actual scan responses are the same
--- PASS: TestMultiScanPrimaryIndex (0.36s)
=== RUN   TestMultiScanDistinct
2019/12/10 22:39:07 In TestScansDistinct()
2019/12/10 22:39:07 

--------- Composite Index with 2 fields ---------
2019/12/10 22:39:07 
--- ScanAllNoFilter ---
2019/12/10 22:39:07 distinct = true
2019/12/10 22:39:08 Using n1ql client
2019/12/10 22:39:08 Expected and Actual scan responses are the same
2019/12/10 22:39:08 
--- ScanAllFilterNil ---
2019/12/10 22:39:08 distinct = true
2019/12/10 22:39:08 Using n1ql client
2019/12/10 22:39:09 Expected and Actual scan responses are the same
2019/12/10 22:39:09 
--- ScanAll_AllFiltersNil ---
2019/12/10 22:39:09 distinct = true
2019/12/10 22:39:09 Using n1ql client
2019/12/10 22:39:09 Expected and Actual scan responses are the same
2019/12/10 22:39:09 
--- SingleSeek ---
2019/12/10 22:39:09 distinct = true
2019/12/10 22:39:10 Using n1ql client
2019/12/10 22:39:10 Expected and Actual scan responses are the same
2019/12/10 22:39:10 
--- MultipleSeek ---
2019/12/10 22:39:10 distinct = true
2019/12/10 22:39:10 Using n1ql client
2019/12/10 22:39:10 Expected and Actual scan responses are the same
2019/12/10 22:39:10 
--- SimpleRange ---
2019/12/10 22:39:10 distinct = true
2019/12/10 22:39:11 Using n1ql client
2019/12/10 22:39:11 Expected and Actual scan responses are the same
2019/12/10 22:39:11 
--- NonOverlappingRanges ---
2019/12/10 22:39:11 distinct = true
2019/12/10 22:39:11 Using n1ql client
2019/12/10 22:39:11 Expected and Actual scan responses are the same
2019/12/10 22:39:11 
--- OverlappingRanges ---
2019/12/10 22:39:11 distinct = true
2019/12/10 22:39:12 Using n1ql client
2019/12/10 22:39:12 Expected and Actual scan responses are the same
2019/12/10 22:39:12 
--- NonOverlappingFilters ---
2019/12/10 22:39:12 distinct = true
2019/12/10 22:39:12 Using n1ql client
2019/12/10 22:39:12 Expected and Actual scan responses are the same
2019/12/10 22:39:12 
--- OverlappingFilters ---
2019/12/10 22:39:12 distinct = true
2019/12/10 22:39:13 Using n1ql client
2019/12/10 22:39:13 Expected and Actual scan responses are the same
2019/12/10 22:39:13 
--- BoundaryFilters ---
2019/12/10 22:39:13 distinct = true
2019/12/10 22:39:14 Using n1ql client
2019/12/10 22:39:14 Expected and Actual scan responses are the same
2019/12/10 22:39:14 
--- SeekAndFilters_NonOverlapping ---
2019/12/10 22:39:14 distinct = true
2019/12/10 22:39:14 Using n1ql client
2019/12/10 22:39:14 Expected and Actual scan responses are the same
2019/12/10 22:39:14 
--- SeekAndFilters_Overlapping ---
2019/12/10 22:39:14 distinct = true
2019/12/10 22:39:15 Using n1ql client
2019/12/10 22:39:15 Expected and Actual scan responses are the same
2019/12/10 22:39:15 
--- SimpleRangeLowUnbounded ---
2019/12/10 22:39:15 distinct = false
2019/12/10 22:39:15 Using n1ql client
2019/12/10 22:39:15 Expected and Actual scan responses are the same
2019/12/10 22:39:15 
--- SimpleRangeHighUnbounded ---
2019/12/10 22:39:15 distinct = false
2019/12/10 22:39:16 Using n1ql client
2019/12/10 22:39:16 Expected and Actual scan responses are the same
2019/12/10 22:39:16 
--- SimpleRangeMultipleUnbounded ---
2019/12/10 22:39:16 distinct = false
2019/12/10 22:39:16 Using n1ql client
2019/12/10 22:39:16 Expected and Actual scan responses are the same
2019/12/10 22:39:16 
--- FiltersWithUnbounded ---
2019/12/10 22:39:16 distinct = false
2019/12/10 22:39:17 Using n1ql client
2019/12/10 22:39:17 Expected and Actual scan responses are the same
2019/12/10 22:39:17 
--- FiltersLowGreaterThanHigh ---
2019/12/10 22:39:17 distinct = false
2019/12/10 22:39:18 Using n1ql client
2019/12/10 22:39:18 Expected and Actual scan responses are the same
2019/12/10 22:39:18 

--------- Simple Index with 1 field ---------
2019/12/10 22:39:18 
--- SingleIndexSimpleRange ---
2019/12/10 22:39:18 distinct = true
2019/12/10 22:39:18 Using n1ql client
2019/12/10 22:39:18 Expected and Actual scan responses are the same
2019/12/10 22:39:18 
--- SingleIndex_SimpleRanges_NonOverlapping ---
2019/12/10 22:39:18 distinct = true
2019/12/10 22:39:18 Using n1ql client
2019/12/10 22:39:19 Expected and Actual scan responses are the same
2019/12/10 22:39:19 
--- SingleIndex_SimpleRanges_Overlapping ---
2019/12/10 22:39:19 distinct = true
2019/12/10 22:39:19 Using n1ql client
2019/12/10 22:39:19 Expected and Actual scan responses are the same
2019/12/10 22:39:19 

--------- Composite Index with 3 fields ---------
2019/12/10 22:39:19 
--- ScanAllNoFilter ---
2019/12/10 22:39:19 distinct = true
2019/12/10 22:39:20 Using n1ql client
2019/12/10 22:39:20 Expected and Actual scan responses are the same
2019/12/10 22:39:20 
--- ScanAllFilterNil ---
2019/12/10 22:39:20 distinct = true
2019/12/10 22:39:20 Using n1ql client
2019/12/10 22:39:21 Expected and Actual scan responses are the same
2019/12/10 22:39:21 
--- ScanAll_AllFiltersNil ---
2019/12/10 22:39:21 distinct = true
2019/12/10 22:39:21 Using n1ql client
2019/12/10 22:39:21 Expected and Actual scan responses are the same
2019/12/10 22:39:21 
--- 3FieldsSingleSeek ---
2019/12/10 22:39:21 distinct = true
2019/12/10 22:39:22 Using n1ql client
2019/12/10 22:39:22 Expected and Actual scan responses are the same
2019/12/10 22:39:22 
--- 3FieldsMultipleSeeks ---
2019/12/10 22:39:22 distinct = true
2019/12/10 22:39:22 Using n1ql client
2019/12/10 22:39:22 Expected and Actual scan responses are the same
2019/12/10 22:39:22 
--- 3FieldsMultipleSeeks_Identical ---
2019/12/10 22:39:22 distinct = true
2019/12/10 22:39:23 Using n1ql client
2019/12/10 22:39:23 Expected and Actual scan responses are the same
--- PASS: TestMultiScanDistinct (15.64s)
=== RUN   TestMultiScanProjection
2019/12/10 22:39:23 In TestMultiScanProjection()
2019/12/10 22:39:23 

--------- Composite Index with 2 fields ---------
2019/12/10 22:39:23 
--- ScanAllNoFilter ---
2019/12/10 22:39:23 distinct = true
2019/12/10 22:39:23 Using n1ql client
2019/12/10 22:39:24 Expected and Actual scan responses are the same
2019/12/10 22:39:24 
--- ScanAllFilterNil ---
2019/12/10 22:39:24 distinct = true
2019/12/10 22:39:24 Using n1ql client
2019/12/10 22:39:24 Expected and Actual scan responses are the same
2019/12/10 22:39:24 
--- ScanAll_AllFiltersNil ---
2019/12/10 22:39:24 distinct = true
2019/12/10 22:39:25 Using n1ql client
2019/12/10 22:39:25 Expected and Actual scan responses are the same
2019/12/10 22:39:25 
--- SingleSeek ---
2019/12/10 22:39:25 distinct = true
2019/12/10 22:39:25 Using n1ql client
2019/12/10 22:39:25 Expected and Actual scan responses are the same
2019/12/10 22:39:25 
--- MultipleSeek ---
2019/12/10 22:39:25 distinct = true
2019/12/10 22:39:26 Using n1ql client
2019/12/10 22:39:26 Expected and Actual scan responses are the same
2019/12/10 22:39:26 
--- SimpleRange ---
2019/12/10 22:39:26 distinct = true
2019/12/10 22:39:26 Using n1ql client
2019/12/10 22:39:26 Expected and Actual scan responses are the same
2019/12/10 22:39:26 
--- NonOverlappingRanges ---
2019/12/10 22:39:26 distinct = true
2019/12/10 22:39:27 Using n1ql client
2019/12/10 22:39:27 Expected and Actual scan responses are the same
2019/12/10 22:39:27 
--- OverlappingRanges ---
2019/12/10 22:39:27 distinct = true
2019/12/10 22:39:27 Using n1ql client
2019/12/10 22:39:28 Expected and Actual scan responses are the same
2019/12/10 22:39:28 
--- NonOverlappingFilters ---
2019/12/10 22:39:28 distinct = true
2019/12/10 22:39:28 Using n1ql client
2019/12/10 22:39:28 Expected and Actual scan responses are the same
2019/12/10 22:39:28 
--- OverlappingFilters ---
2019/12/10 22:39:28 distinct = true
2019/12/10 22:39:28 Using n1ql client
2019/12/10 22:39:29 Expected and Actual scan responses are the same
2019/12/10 22:39:29 
--- BoundaryFilters ---
2019/12/10 22:39:29 distinct = true
2019/12/10 22:39:29 Using n1ql client
2019/12/10 22:39:29 Expected and Actual scan responses are the same
2019/12/10 22:39:29 
--- SeekAndFilters_NonOverlapping ---
2019/12/10 22:39:29 distinct = true
2019/12/10 22:39:30 Using n1ql client
2019/12/10 22:39:30 Expected and Actual scan responses are the same
2019/12/10 22:39:30 
--- SeekAndFilters_Overlapping ---
2019/12/10 22:39:30 distinct = true
2019/12/10 22:39:30 Using n1ql client
2019/12/10 22:39:30 Expected and Actual scan responses are the same
2019/12/10 22:39:30 
--- SimpleRangeLowUnbounded ---
2019/12/10 22:39:30 distinct = false
2019/12/10 22:39:31 Using n1ql client
2019/12/10 22:39:31 Expected and Actual scan responses are the same
2019/12/10 22:39:31 
--- SimpleRangeHighUnbounded ---
2019/12/10 22:39:31 distinct = false
2019/12/10 22:39:31 Using n1ql client
2019/12/10 22:39:31 Expected and Actual scan responses are the same
2019/12/10 22:39:31 
--- SimpleRangeMultipleUnbounded ---
2019/12/10 22:39:31 distinct = false
2019/12/10 22:39:32 Using n1ql client
2019/12/10 22:39:32 Expected and Actual scan responses are the same
2019/12/10 22:39:32 
--- FiltersWithUnbounded ---
2019/12/10 22:39:32 distinct = false
2019/12/10 22:39:32 Using n1ql client
2019/12/10 22:39:33 Expected and Actual scan responses are the same
2019/12/10 22:39:33 
--- FiltersLowGreaterThanHigh ---
2019/12/10 22:39:33 distinct = false
2019/12/10 22:39:33 Using n1ql client
2019/12/10 22:39:33 Expected and Actual scan responses are the same
2019/12/10 22:39:33 

--------- Simple Index with 1 field ---------
2019/12/10 22:39:33 
--- SingleIndexSimpleRange ---
2019/12/10 22:39:33 distinct = true
2019/12/10 22:39:33 Using n1ql client
2019/12/10 22:39:33 Expected and Actual scan responses are the same
2019/12/10 22:39:33 
--- SingleIndex_SimpleRanges_NonOverlapping ---
2019/12/10 22:39:33 distinct = true
2019/12/10 22:39:34 Using n1ql client
2019/12/10 22:39:34 Expected and Actual scan responses are the same
2019/12/10 22:39:34 
--- SingleIndex_SimpleRanges_Overlapping ---
2019/12/10 22:39:34 distinct = true
2019/12/10 22:39:34 Using n1ql client
2019/12/10 22:39:35 Expected and Actual scan responses are the same
2019/12/10 22:39:35 

--------- Composite Index with 3 fields ---------
2019/12/10 22:39:35 
--- ScanAllNoFilter ---
2019/12/10 22:39:35 distinct = true
2019/12/10 22:39:35 Using n1ql client
2019/12/10 22:39:35 Expected and Actual scan responses are the same
2019/12/10 22:39:35 
--- ScanAllFilterNil ---
2019/12/10 22:39:35 distinct = true
2019/12/10 22:39:36 Using n1ql client
2019/12/10 22:39:36 Expected and Actual scan responses are the same
2019/12/10 22:39:36 
--- ScanAll_AllFiltersNil ---
2019/12/10 22:39:36 distinct = true
2019/12/10 22:39:37 Using n1ql client
2019/12/10 22:39:37 Expected and Actual scan responses are the same
2019/12/10 22:39:37 
--- 3FieldsSingleSeek ---
2019/12/10 22:39:37 distinct = true
2019/12/10 22:39:37 Using n1ql client
2019/12/10 22:39:37 Expected and Actual scan responses are the same
2019/12/10 22:39:37 
--- 3FieldsMultipleSeeks ---
2019/12/10 22:39:37 distinct = true
2019/12/10 22:39:38 Using n1ql client
2019/12/10 22:39:38 Expected and Actual scan responses are the same
2019/12/10 22:39:38 
--- 3FieldsMultipleSeeks_Identical ---
2019/12/10 22:39:38 distinct = true
2019/12/10 22:39:38 Using n1ql client
2019/12/10 22:39:38 Expected and Actual scan responses are the same
2019/12/10 22:39:38 indexes are: index_company, index_companyname, index_company_name_age, index_company_name_age_address, index_company_name_age_address_friends
2019/12/10 22:39:38 fields are: [company], [company name], [company name age], [company name age address], [company name age address friends]
2019/12/10 22:39:38 
--- SingleIndexProjectFirst ---
2019/12/10 22:39:38 distinct = true
2019/12/10 22:39:39 Using n1ql client
2019/12/10 22:39:39 Expected and Actual scan responses are the same
2019/12/10 22:39:39 
--- 2FieldIndexProjectSecond ---
2019/12/10 22:39:39 distinct = true
2019/12/10 22:39:39 Using n1ql client
2019/12/10 22:39:39 Expected and Actual scan responses are the same
2019/12/10 22:39:39 
--- 3FieldIndexProjectThird ---
2019/12/10 22:39:39 distinct = true
2019/12/10 22:39:40 Using n1ql client
2019/12/10 22:39:40 Expected and Actual scan responses are the same
2019/12/10 22:39:40 
--- 4FieldIndexProjectFourth ---
2019/12/10 22:39:40 distinct = true
2019/12/10 22:39:42 Using n1ql client
2019/12/10 22:39:42 Expected and Actual scan responses are the same
2019/12/10 22:39:42 
--- 5FieldIndexProjectFifth ---
2019/12/10 22:39:42 distinct = true
2019/12/10 22:39:46 Using n1ql client
2019/12/10 22:39:46 Expected and Actual scan responses are the same
2019/12/10 22:39:46 
--- 2FieldIndexProjectTwo ---
2019/12/10 22:39:46 distinct = true
2019/12/10 22:39:47 Using n1ql client
2019/12/10 22:39:47 Expected and Actual scan responses are the same
2019/12/10 22:39:47 
--- 3FieldIndexProjectTwo ---
2019/12/10 22:39:47 distinct = true
2019/12/10 22:39:47 Using n1ql client
2019/12/10 22:39:47 Expected and Actual scan responses are the same
2019/12/10 22:39:47 
--- 3FieldIndexProjectTwo ---
2019/12/10 22:39:47 distinct = true
2019/12/10 22:39:48 Using n1ql client
2019/12/10 22:39:48 Expected and Actual scan responses are the same
2019/12/10 22:39:48 
--- 3FieldIndexProjectTwo ---
2019/12/10 22:39:48 distinct = true
2019/12/10 22:39:48 Using n1ql client
2019/12/10 22:39:48 Expected and Actual scan responses are the same
2019/12/10 22:39:48 
--- 4FieldIndexProjectTwo ---
2019/12/10 22:39:48 distinct = true
2019/12/10 22:39:50 Using n1ql client
2019/12/10 22:39:50 Expected and Actual scan responses are the same
2019/12/10 22:39:50 
--- 4FieldIndexProjectTwo ---
2019/12/10 22:39:50 distinct = true
2019/12/10 22:39:52 Using n1ql client
2019/12/10 22:39:52 Expected and Actual scan responses are the same
2019/12/10 22:39:52 
--- 4FieldIndexProjectTwo ---
2019/12/10 22:39:52 distinct = true
2019/12/10 22:39:54 Using n1ql client
2019/12/10 22:39:54 Expected and Actual scan responses are the same
2019/12/10 22:39:54 
--- 4FieldIndexProjectTwo ---
2019/12/10 22:39:54 distinct = true
2019/12/10 22:39:56 Using n1ql client
2019/12/10 22:39:56 Expected and Actual scan responses are the same
2019/12/10 22:39:56 
--- 4FieldIndexProjectTwo ---
2019/12/10 22:39:56 distinct = true
2019/12/10 22:39:58 Using n1ql client
2019/12/10 22:39:59 Expected and Actual scan responses are the same
2019/12/10 22:39:59 
--- 5FieldIndexProjectTwo ---
2019/12/10 22:39:59 distinct = true
2019/12/10 22:40:03 Using n1ql client
2019/12/10 22:40:03 Expected and Actual scan responses are the same
2019/12/10 22:40:03 
--- 5FieldIndexProjectTwo ---
2019/12/10 22:40:03 distinct = true
2019/12/10 22:40:07 Using n1ql client
2019/12/10 22:40:07 Expected and Actual scan responses are the same
2019/12/10 22:40:07 
--- 5FieldIndexProjectTwo ---
2019/12/10 22:40:07 distinct = true
2019/12/10 22:40:11 Using n1ql client
2019/12/10 22:40:11 Expected and Actual scan responses are the same
2019/12/10 22:40:11 
--- 5FieldIndexProjectTwo ---
2019/12/10 22:40:11 distinct = true
2019/12/10 22:40:15 Using n1ql client
2019/12/10 22:40:15 Expected and Actual scan responses are the same
2019/12/10 22:40:15 
--- 5FieldIndexProjectThree ---
2019/12/10 22:40:15 distinct = true
2019/12/10 22:40:19 Using n1ql client
2019/12/10 22:40:19 Expected and Actual scan responses are the same
2019/12/10 22:40:19 
--- 5FieldIndexProjectFour ---
2019/12/10 22:40:19 distinct = true
2019/12/10 22:40:24 Using n1ql client
2019/12/10 22:40:24 Expected and Actual scan responses are the same
2019/12/10 22:40:24 
--- 5FieldIndexProjectAll ---
2019/12/10 22:40:24 distinct = true
2019/12/10 22:40:28 Using n1ql client
2019/12/10 22:40:28 Expected and Actual scan responses are the same
2019/12/10 22:40:28 
--- 5FieldIndexProjectAlternate ---
2019/12/10 22:40:28 distinct = true
2019/12/10 22:40:32 Using n1ql client
2019/12/10 22:40:32 Expected and Actual scan responses are the same
2019/12/10 22:40:32 
--- 5FieldIndexProjectEmptyEntryKeys ---
2019/12/10 22:40:32 distinct = true
2019/12/10 22:40:36 Using n1ql client
2019/12/10 22:40:36 Expected and Actual scan responses are the same
--- PASS: TestMultiScanProjection (73.21s)
=== RUN   TestMultiScanRestAPI
2019/12/10 22:40:36 In TestMultiScanRestAPI()
2019/12/10 22:40:36 In DropAllSecondaryIndexes()
2019/12/10 22:40:36 Index found:  addressidx
2019/12/10 22:40:36 Dropped index addressidx
2019/12/10 22:40:36 Index found:  index_company_name_age
2019/12/10 22:40:37 Dropped index index_company_name_age
2019/12/10 22:40:37 Index found:  index_company
2019/12/10 22:40:37 Dropped index index_company
2019/12/10 22:40:37 Index found:  index_companyname
2019/12/10 22:40:37 Dropped index index_companyname
2019/12/10 22:40:37 Index found:  index_primary
2019/12/10 22:40:37 Dropped index index_primary
2019/12/10 22:40:37 Index found:  index_company_name_age_address
2019/12/10 22:40:38 Dropped index index_company_name_age_address
2019/12/10 22:40:38 Index found:  index_company_name_age_address_friends
2019/12/10 22:40:38 Dropped index index_company_name_age_address_friends
2019/12/10 22:40:43 Created the secondary index index_companyname. Waiting for it become active
2019/12/10 22:40:43 Index is now active
2019/12/10 22:40:43 GET all indexes
2019/12/10 22:40:43 200 OK
2019/12/10 22:40:43 getscans status : 200 OK
2019/12/10 22:40:43 number of entries 337
2019/12/10 22:40:43 Status : 200 OK
2019/12/10 22:40:43 Result from multiscancount API = 337
--- PASS: TestMultiScanRestAPI (6.76s)
=== RUN   TestMultiScanPrimaryIndexVariations
2019/12/10 22:40:43 In TestMultiScanPrimaryIndexVariations()
2019/12/10 22:40:48 Created the secondary index index_pi. Waiting for it become active
2019/12/10 22:40:48 Index is now active
2019/12/10 22:40:48 
--- No Overlap ---
2019/12/10 22:40:48 Using n1ql client
2019/12/10 22:40:48 Expected and Actual scan responses are the same
2019/12/10 22:40:48 
--- Proper Overlap ---
2019/12/10 22:40:48 Using n1ql client
2019/12/10 22:40:48 Expected and Actual scan responses are the same
2019/12/10 22:40:48 
--- Low Boundary Overlap ---
2019/12/10 22:40:48 Using n1ql client
2019/12/10 22:40:48 Expected and Actual scan responses are the same
2019/12/10 22:40:48 
--- Complex Overlaps ---
2019/12/10 22:40:48 Using n1ql client
2019/12/10 22:40:48 Expected and Actual scan responses are the same
2019/12/10 22:40:48 
--- Multiple Equal Overlaps ---
2019/12/10 22:40:48 Using n1ql client
2019/12/10 22:40:48 Expected and Actual scan responses are the same
2019/12/10 22:40:48 
--- Boundary and Subset Overlaps ---
2019/12/10 22:40:48 Using n1ql client
2019/12/10 22:40:48 Expected and Actual scan responses are the same
2019/12/10 22:40:48 
--- Point Overlaps ---
2019/12/10 22:40:48 Using n1ql client
2019/12/10 22:40:48 Expected and Actual scan responses are the same
2019/12/10 22:40:48 
--- Boundary and Point Overlaps ---
2019/12/10 22:40:48 Using n1ql client
2019/12/10 22:40:48 Expected and Actual scan responses are the same
2019/12/10 22:40:48 
--- Primary index range null ---
2019/12/10 22:40:48 Using n1ql client
2019/12/10 22:40:48 Expected and Actual scan responses are the same
2019/12/10 22:40:48 Dropping the secondary index index_pi
2019/12/10 22:40:48 Index dropped
--- PASS: TestMultiScanPrimaryIndexVariations (5.54s)
=== RUN   TestMultiScanDescSetup
2019/12/10 22:40:48 In TestMultiScanDescSetup()
2019/12/10 22:40:48 In DropAllSecondaryIndexes()
2019/12/10 22:40:48 Index found:  index_companyname
2019/12/10 22:40:49 Dropped index index_companyname
2019/12/10 22:40:54 Created the secondary index index_companyname_desc. Waiting for it become active
2019/12/10 22:40:54 Index is now active
2019/12/10 22:40:59 Created the secondary index index_company_desc. Waiting for it become active
2019/12/10 22:40:59 Index is now active
2019/12/10 22:41:04 Created the secondary index index_company_name_age_desc. Waiting for it become active
2019/12/10 22:41:04 Index is now active
--- PASS: TestMultiScanDescSetup (15.78s)
=== RUN   TestMultiScanDescScenarios
2019/12/10 22:41:04 In TestMultiScanDescScenarios()
2019/12/10 22:41:04 

--------- Composite Index with 2 fields ---------
2019/12/10 22:41:04 
--- ScanAllNoFilter ---
2019/12/10 22:41:04 distinct = false
2019/12/10 22:41:05 Using n1ql client
2019/12/10 22:41:05 Expected and Actual scan responses are the same
2019/12/10 22:41:05 
--- ScanAllFilterNil ---
2019/12/10 22:41:05 distinct = false
2019/12/10 22:41:05 Using n1ql client
2019/12/10 22:41:06 Expected and Actual scan responses are the same
2019/12/10 22:41:06 
--- ScanAll_AllFiltersNil ---
2019/12/10 22:41:06 distinct = false
2019/12/10 22:41:06 Using n1ql client
2019/12/10 22:41:06 Expected and Actual scan responses are the same
2019/12/10 22:41:06 
--- SingleSeek ---
2019/12/10 22:41:06 distinct = false
2019/12/10 22:41:07 Using n1ql client
2019/12/10 22:41:07 Expected and Actual scan responses are the same
2019/12/10 22:41:07 
--- MultipleSeek ---
2019/12/10 22:41:07 distinct = false
2019/12/10 22:41:07 Using n1ql client
2019/12/10 22:41:07 Expected and Actual scan responses are the same
2019/12/10 22:41:07 
--- SimpleRange ---
2019/12/10 22:41:07 distinct = false
2019/12/10 22:41:08 Using n1ql client
2019/12/10 22:41:08 Expected and Actual scan responses are the same
2019/12/10 22:41:08 
--- NonOverlappingRanges ---
2019/12/10 22:41:08 distinct = false
2019/12/10 22:41:08 Using n1ql client
2019/12/10 22:41:08 Expected and Actual scan responses are the same
2019/12/10 22:41:08 
--- OverlappingRanges ---
2019/12/10 22:41:08 distinct = false
2019/12/10 22:41:09 Using n1ql client
2019/12/10 22:41:09 Expected and Actual scan responses are the same
2019/12/10 22:41:09 
--- NonOverlappingFilters ---
2019/12/10 22:41:09 distinct = false
2019/12/10 22:41:09 Using n1ql client
2019/12/10 22:41:10 Expected and Actual scan responses are the same
2019/12/10 22:41:10 
--- OverlappingFilters ---
2019/12/10 22:41:10 distinct = false
2019/12/10 22:41:10 Using n1ql client
2019/12/10 22:41:10 Expected and Actual scan responses are the same
2019/12/10 22:41:10 
--- BoundaryFilters ---
2019/12/10 22:41:10 distinct = false
2019/12/10 22:41:11 Using n1ql client
2019/12/10 22:41:11 Expected and Actual scan responses are the same
2019/12/10 22:41:11 
--- SeekAndFilters_NonOverlapping ---
2019/12/10 22:41:11 distinct = false
2019/12/10 22:41:11 Using n1ql client
2019/12/10 22:41:11 Expected and Actual scan responses are the same
2019/12/10 22:41:11 
--- SeekAndFilters_Overlapping ---
2019/12/10 22:41:11 distinct = false
2019/12/10 22:41:12 Using n1ql client
2019/12/10 22:41:12 Expected and Actual scan responses are the same
2019/12/10 22:41:12 
--- SimpleRangeLowUnbounded ---
2019/12/10 22:41:12 distinct = false
2019/12/10 22:41:12 Using n1ql client
2019/12/10 22:41:12 Expected and Actual scan responses are the same
2019/12/10 22:41:12 
--- SimpleRangeHighUnbounded ---
2019/12/10 22:41:12 distinct = false
2019/12/10 22:41:13 Using n1ql client
2019/12/10 22:41:13 Expected and Actual scan responses are the same
2019/12/10 22:41:13 
--- SimpleRangeMultipleUnbounded ---
2019/12/10 22:41:13 distinct = false
2019/12/10 22:41:13 Using n1ql client
2019/12/10 22:41:13 Expected and Actual scan responses are the same
2019/12/10 22:41:13 
--- FiltersWithUnbounded ---
2019/12/10 22:41:13 distinct = false
2019/12/10 22:41:14 Using n1ql client
2019/12/10 22:41:14 Expected and Actual scan responses are the same
2019/12/10 22:41:14 
--- FiltersLowGreaterThanHigh ---
2019/12/10 22:41:14 distinct = false
2019/12/10 22:41:15 Using n1ql client
2019/12/10 22:41:15 Expected and Actual scan responses are the same
2019/12/10 22:41:15 

--------- Simple Index with 1 field ---------
2019/12/10 22:41:15 
--- SingleIndexSimpleRange ---
2019/12/10 22:41:15 distinct = false
2019/12/10 22:41:15 Using n1ql client
2019/12/10 22:41:15 Expected and Actual scan responses are the same
2019/12/10 22:41:15 
--- SingleIndex_SimpleRanges_NonOverlapping ---
2019/12/10 22:41:15 distinct = false
2019/12/10 22:41:15 Using n1ql client
2019/12/10 22:41:16 Expected and Actual scan responses are the same
2019/12/10 22:41:16 
--- SingleIndex_SimpleRanges_Overlapping ---
2019/12/10 22:41:16 distinct = false
2019/12/10 22:41:16 Using n1ql client
2019/12/10 22:41:16 Expected and Actual scan responses are the same
2019/12/10 22:41:16 

--------- Composite Index with 3 fields ---------
2019/12/10 22:41:16 
--- ScanAllNoFilter ---
2019/12/10 22:41:16 distinct = false
2019/12/10 22:41:17 Using n1ql client
2019/12/10 22:41:17 Expected and Actual scan responses are the same
2019/12/10 22:41:17 
--- ScanAllFilterNil ---
2019/12/10 22:41:17 distinct = false
2019/12/10 22:41:17 Using n1ql client
2019/12/10 22:41:18 Expected and Actual scan responses are the same
2019/12/10 22:41:18 
--- ScanAll_AllFiltersNil ---
2019/12/10 22:41:18 distinct = false
2019/12/10 22:41:18 Using n1ql client
2019/12/10 22:41:18 Expected and Actual scan responses are the same
2019/12/10 22:41:18 
--- 3FieldsSingleSeek ---
2019/12/10 22:41:18 distinct = false
2019/12/10 22:41:19 Using n1ql client
2019/12/10 22:41:19 Expected and Actual scan responses are the same
2019/12/10 22:41:19 
--- 3FieldsMultipleSeeks ---
2019/12/10 22:41:19 distinct = false
2019/12/10 22:41:19 Using n1ql client
2019/12/10 22:41:19 Expected and Actual scan responses are the same
2019/12/10 22:41:19 
--- 3FieldsMultipleSeeks_Identical ---
2019/12/10 22:41:19 distinct = false
2019/12/10 22:41:20 Using n1ql client
2019/12/10 22:41:20 Expected and Actual scan responses are the same
2019/12/10 22:41:20 

--------- New scenarios ---------
2019/12/10 22:41:20 
--- CompIndexHighUnbounded1 ---
2019/12/10 22:41:20 
--- Multi Scan 0 ---
2019/12/10 22:41:20 distinct = false
2019/12/10 22:41:20 Using n1ql client
2019/12/10 22:41:20 Expected and Actual scan responses are the same
2019/12/10 22:41:20 
--- Multi Scan 1 ---
2019/12/10 22:41:20 distinct = false
2019/12/10 22:41:21 Using n1ql client
2019/12/10 22:41:21 Expected and Actual scan responses are the same
2019/12/10 22:41:21 
--- Multi Scan 2 ---
2019/12/10 22:41:21 distinct = false
2019/12/10 22:41:21 Using n1ql client
2019/12/10 22:41:21 Expected and Actual scan responses are the same
2019/12/10 22:41:21 
--- CompIndexHighUnbounded2 ---
2019/12/10 22:41:21 
--- Multi Scan 0 ---
2019/12/10 22:41:21 distinct = false
2019/12/10 22:41:22 Using n1ql client
2019/12/10 22:41:22 Expected and Actual scan responses are the same
2019/12/10 22:41:22 
--- Multi Scan 1 ---
2019/12/10 22:41:22 distinct = false
2019/12/10 22:41:22 Using n1ql client
2019/12/10 22:41:23 Expected and Actual scan responses are the same
2019/12/10 22:41:23 
--- Multi Scan 2 ---
2019/12/10 22:41:23 distinct = false
2019/12/10 22:41:23 Using n1ql client
2019/12/10 22:41:23 Expected and Actual scan responses are the same
2019/12/10 22:41:23 
--- CompIndexHighUnbounded3 ---
2019/12/10 22:41:23 
--- Multi Scan 0 ---
2019/12/10 22:41:23 distinct = false
2019/12/10 22:41:24 Using n1ql client
2019/12/10 22:41:24 Expected and Actual scan responses are the same
2019/12/10 22:41:24 
--- CompIndexHighUnbounded4 ---
2019/12/10 22:41:24 
--- Multi Scan 0 ---
2019/12/10 22:41:24 distinct = false
2019/12/10 22:41:24 Using n1ql client
2019/12/10 22:41:24 Expected and Actual scan responses are the same
2019/12/10 22:41:24 
--- CompIndexHighUnbounded5 ---
2019/12/10 22:41:24 
--- Multi Scan 0 ---
2019/12/10 22:41:24 distinct = false
2019/12/10 22:41:25 Using n1ql client
2019/12/10 22:41:25 Expected and Actual scan responses are the same
2019/12/10 22:41:25 
--- SeekBoundaries ---
2019/12/10 22:41:25 
--- Multi Scan 0 ---
2019/12/10 22:41:25 distinct = false
2019/12/10 22:41:26 Using n1ql client
2019/12/10 22:41:26 Expected and Actual scan responses are the same
2019/12/10 22:41:26 
--- Multi Scan 1 ---
2019/12/10 22:41:26 distinct = false
2019/12/10 22:41:26 Using n1ql client
2019/12/10 22:41:26 Expected and Actual scan responses are the same
2019/12/10 22:41:26 
--- Multi Scan 2 ---
2019/12/10 22:41:26 distinct = false
2019/12/10 22:41:26 Using n1ql client
2019/12/10 22:41:27 Expected and Actual scan responses are the same
2019/12/10 22:41:27 
--- Multi Scan 3 ---
2019/12/10 22:41:27 distinct = false
2019/12/10 22:41:27 Using n1ql client
2019/12/10 22:41:27 Expected and Actual scan responses are the same
2019/12/10 22:41:27 
--- Multi Scan 4 ---
2019/12/10 22:41:27 distinct = false
2019/12/10 22:41:28 Using n1ql client
2019/12/10 22:41:28 Expected and Actual scan responses are the same
2019/12/10 22:41:28 
--- Multi Scan 5 ---
2019/12/10 22:41:28 distinct = false
2019/12/10 22:41:28 Using n1ql client
2019/12/10 22:41:28 Expected and Actual scan responses are the same
2019/12/10 22:41:28 
--- Multi Scan 6 ---
2019/12/10 22:41:28 distinct = false
2019/12/10 22:41:29 Using n1ql client
2019/12/10 22:41:29 Expected and Actual scan responses are the same
2019/12/10 22:41:29 
--- Multi Scan 7 ---
2019/12/10 22:41:29 distinct = false
2019/12/10 22:41:29 Using n1ql client
2019/12/10 22:41:29 Expected and Actual scan responses are the same
--- PASS: TestMultiScanDescScenarios (25.10s)
=== RUN   TestMultiScanDescCount
2019/12/10 22:41:29 In TestMultiScanDescCount()
2019/12/10 22:41:29 

--------- Composite Index with 2 fields ---------
2019/12/10 22:41:29 
--- ScanAllNoFilter ---
2019/12/10 22:41:29 distinct = false
2019/12/10 22:41:30 Using n1ql client
2019/12/10 22:41:30 MultiScanCount = 10002 ExpectedMultiScanCount = 10002
2019/12/10 22:41:30 
--- ScanAllFilterNil ---
2019/12/10 22:41:30 distinct = false
2019/12/10 22:41:30 Using n1ql client
2019/12/10 22:41:31 MultiScanCount = 10002 ExpectedMultiScanCount = 10002
2019/12/10 22:41:31 
--- ScanAll_AllFiltersNil ---
2019/12/10 22:41:31 distinct = false
2019/12/10 22:41:31 Using n1ql client
2019/12/10 22:41:31 MultiScanCount = 10002 ExpectedMultiScanCount = 10002
2019/12/10 22:41:31 
--- SingleSeek ---
2019/12/10 22:41:31 distinct = false
2019/12/10 22:41:32 Using n1ql client
2019/12/10 22:41:32 MultiScanCount = 1 ExpectedMultiScanCount = 1
2019/12/10 22:41:32 
--- MultipleSeek ---
2019/12/10 22:41:32 distinct = false
2019/12/10 22:41:32 Using n1ql client
2019/12/10 22:41:32 MultiScanCount = 2 ExpectedMultiScanCount = 2
2019/12/10 22:41:32 
--- SimpleRange ---
2019/12/10 22:41:32 distinct = false
2019/12/10 22:41:33 Using n1ql client
2019/12/10 22:41:33 MultiScanCount = 2273 ExpectedMultiScanCount = 2273
2019/12/10 22:41:33 
--- NonOverlappingRanges ---
2019/12/10 22:41:33 distinct = false
2019/12/10 22:41:33 Using n1ql client
2019/12/10 22:41:33 MultiScanCount = 4283 ExpectedMultiScanCount = 4283
2019/12/10 22:41:33 
--- OverlappingRanges ---
2019/12/10 22:41:33 distinct = false
2019/12/10 22:41:34 Using n1ql client
2019/12/10 22:41:34 MultiScanCount = 5756 ExpectedMultiScanCount = 5756
2019/12/10 22:41:34 
--- NonOverlappingFilters ---
2019/12/10 22:41:34 distinct = false
2019/12/10 22:41:34 Using n1ql client
2019/12/10 22:41:34 MultiScanCount = 337 ExpectedMultiScanCount = 337
2019/12/10 22:41:34 
--- OverlappingFilters ---
2019/12/10 22:41:34 distinct = false
2019/12/10 22:41:35 Using n1ql client
2019/12/10 22:41:35 MultiScanCount = 2559 ExpectedMultiScanCount = 2559
2019/12/10 22:41:35 
--- BoundaryFilters ---
2019/12/10 22:41:35 distinct = false
2019/12/10 22:41:35 Using n1ql client
2019/12/10 22:41:35 MultiScanCount = 499 ExpectedMultiScanCount = 499
2019/12/10 22:41:35 
--- SeekAndFilters_NonOverlapping ---
2019/12/10 22:41:35 distinct = false
2019/12/10 22:41:36 Using n1ql client
2019/12/10 22:41:36 MultiScanCount = 256 ExpectedMultiScanCount = 256
2019/12/10 22:41:36 
--- SeekAndFilters_Overlapping ---
2019/12/10 22:41:36 distinct = false
2019/12/10 22:41:36 Using n1ql client
2019/12/10 22:41:36 MultiScanCount = 255 ExpectedMultiScanCount = 255
2019/12/10 22:41:36 
--- SimpleRangeLowUnbounded ---
2019/12/10 22:41:36 distinct = false
2019/12/10 22:41:37 Using n1ql client
2019/12/10 22:41:37 MultiScanCount = 5618 ExpectedMultiScanCount = 5618
2019/12/10 22:41:37 
--- SimpleRangeHighUnbounded ---
2019/12/10 22:41:37 distinct = false
2019/12/10 22:41:37 Using n1ql client
2019/12/10 22:41:37 MultiScanCount = 3704 ExpectedMultiScanCount = 3704
2019/12/10 22:41:37 
--- SimpleRangeMultipleUnbounded ---
2019/12/10 22:41:37 distinct = false
2019/12/10 22:41:38 Using n1ql client
2019/12/10 22:41:38 MultiScanCount = 10002 ExpectedMultiScanCount = 10002
2019/12/10 22:41:38 
--- FiltersWithUnbounded ---
2019/12/10 22:41:38 distinct = false
2019/12/10 22:41:39 Using n1ql client
2019/12/10 22:41:39 MultiScanCount = 3173 ExpectedMultiScanCount = 3173
2019/12/10 22:41:39 
--- FiltersLowGreaterThanHigh ---
2019/12/10 22:41:39 distinct = false
2019/12/10 22:41:39 Using n1ql client
2019/12/10 22:41:39 MultiScanCount = 418 ExpectedMultiScanCount = 418
2019/12/10 22:41:39 

--------- Simple Index with 1 field ---------
2019/12/10 22:41:39 
--- SingleIndexSimpleRange ---
2019/12/10 22:41:39 distinct = false
2019/12/10 22:41:40 Using n1ql client
2019/12/10 22:41:40 MultiScanCount = 2273 ExpectedMultiScanCount = 2273
2019/12/10 22:41:40 
--- SingleIndex_SimpleRanges_NonOverlapping ---
2019/12/10 22:41:40 distinct = false
2019/12/10 22:41:40 Using n1ql client
2019/12/10 22:41:40 MultiScanCount = 7140 ExpectedMultiScanCount = 7140
2019/12/10 22:41:40 
--- SingleIndex_SimpleRanges_Overlapping ---
2019/12/10 22:41:40 distinct = false
2019/12/10 22:41:41 Using n1ql client
2019/12/10 22:41:41 MultiScanCount = 8701 ExpectedMultiScanCount = 8701
2019/12/10 22:41:41 

--------- Composite Index with 3 fields ---------
2019/12/10 22:41:41 
--- ScanAllNoFilter ---
2019/12/10 22:41:41 distinct = false
2019/12/10 22:41:41 Using n1ql client
2019/12/10 22:41:41 MultiScanCount = 10002 ExpectedMultiScanCount = 10002
2019/12/10 22:41:41 
--- ScanAllFilterNil ---
2019/12/10 22:41:41 distinct = false
2019/12/10 22:41:42 Using n1ql client
2019/12/10 22:41:42 MultiScanCount = 10002 ExpectedMultiScanCount = 10002
2019/12/10 22:41:42 
--- ScanAll_AllFiltersNil ---
2019/12/10 22:41:42 distinct = false
2019/12/10 22:41:43 Using n1ql client
2019/12/10 22:41:43 MultiScanCount = 10002 ExpectedMultiScanCount = 10002
2019/12/10 22:41:43 
--- 3FieldsSingleSeek ---
2019/12/10 22:41:43 distinct = false
2019/12/10 22:41:43 Using n1ql client
2019/12/10 22:41:43 MultiScanCount = 1 ExpectedMultiScanCount = 1
2019/12/10 22:41:43 
--- 3FieldsMultipleSeeks ---
2019/12/10 22:41:43 distinct = false
2019/12/10 22:41:44 Using n1ql client
2019/12/10 22:41:44 MultiScanCount = 3 ExpectedMultiScanCount = 3
2019/12/10 22:41:44 
--- 3FieldsMultipleSeeks_Identical ---
2019/12/10 22:41:44 distinct = false
2019/12/10 22:41:44 Using n1ql client
2019/12/10 22:41:44 MultiScanCount = 2 ExpectedMultiScanCount = 2
2019/12/10 22:41:44 

--------- New scenarios ---------
2019/12/10 22:41:44 
--- CompIndexHighUnbounded1 ---
2019/12/10 22:41:44 
--- Multi Scan 0 ---
2019/12/10 22:41:44 distinct = false
2019/12/10 22:41:45 Using n1ql client
2019/12/10 22:41:45 Using n1ql client
2019/12/10 22:41:45 len(scanResults) = 8 MultiScanCount = 8
2019/12/10 22:41:45 Expected and Actual scan responses are the same
2019/12/10 22:41:45 
--- Multi Scan 1 ---
2019/12/10 22:41:45 distinct = false
2019/12/10 22:41:45 Using n1ql client
2019/12/10 22:41:45 Using n1ql client
2019/12/10 22:41:45 len(scanResults) = 0 MultiScanCount = 0
2019/12/10 22:41:45 Expected and Actual scan responses are the same
2019/12/10 22:41:45 
--- Multi Scan 2 ---
2019/12/10 22:41:45 distinct = false
2019/12/10 22:41:46 Using n1ql client
2019/12/10 22:41:46 Using n1ql client
2019/12/10 22:41:46 len(scanResults) = 9 MultiScanCount = 9
2019/12/10 22:41:46 Expected and Actual scan responses are the same
2019/12/10 22:41:46 
--- CompIndexHighUnbounded2 ---
2019/12/10 22:41:46 
--- Multi Scan 0 ---
2019/12/10 22:41:46 distinct = false
2019/12/10 22:41:46 Using n1ql client
2019/12/10 22:41:46 Using n1ql client
2019/12/10 22:41:46 len(scanResults) = 4138 MultiScanCount = 4138
2019/12/10 22:41:46 Expected and Actual scan responses are the same
2019/12/10 22:41:46 
--- Multi Scan 1 ---
2019/12/10 22:41:46 distinct = false
2019/12/10 22:41:47 Using n1ql client
2019/12/10 22:41:47 Using n1ql client
2019/12/10 22:41:47 len(scanResults) = 2746 MultiScanCount = 2746
2019/12/10 22:41:47 Expected and Actual scan responses are the same
2019/12/10 22:41:47 
--- Multi Scan 2 ---
2019/12/10 22:41:47 distinct = false
2019/12/10 22:41:47 Using n1ql client
2019/12/10 22:41:48 Using n1ql client
2019/12/10 22:41:48 len(scanResults) = 4691 MultiScanCount = 4691
2019/12/10 22:41:48 Expected and Actual scan responses are the same
2019/12/10 22:41:48 
--- CompIndexHighUnbounded3 ---
2019/12/10 22:41:48 
--- Multi Scan 0 ---
2019/12/10 22:41:48 distinct = false
2019/12/10 22:41:48 Using n1ql client
2019/12/10 22:41:48 Using n1ql client
2019/12/10 22:41:48 len(scanResults) = 1329 MultiScanCount = 1329
2019/12/10 22:41:48 Expected and Actual scan responses are the same
2019/12/10 22:41:48 
--- CompIndexHighUnbounded4 ---
2019/12/10 22:41:48 
--- Multi Scan 0 ---
2019/12/10 22:41:48 distinct = false
2019/12/10 22:41:49 Using n1ql client
2019/12/10 22:41:49 Using n1ql client
2019/12/10 22:41:49 len(scanResults) = 5349 MultiScanCount = 5349
2019/12/10 22:41:49 Expected and Actual scan responses are the same
2019/12/10 22:41:49 
--- CompIndexHighUnbounded5 ---
2019/12/10 22:41:49 
--- Multi Scan 0 ---
2019/12/10 22:41:49 distinct = false
2019/12/10 22:41:50 Using n1ql client
2019/12/10 22:41:50 Using n1ql client
2019/12/10 22:41:50 len(scanResults) = 8210 MultiScanCount = 8210
2019/12/10 22:41:50 Expected and Actual scan responses are the same
2019/12/10 22:41:50 
--- SeekBoundaries ---
2019/12/10 22:41:50 
--- Multi Scan 0 ---
2019/12/10 22:41:50 distinct = false
2019/12/10 22:41:50 Using n1ql client
2019/12/10 22:41:50 Using n1ql client
2019/12/10 22:41:51 len(scanResults) = 175 MultiScanCount = 175
2019/12/10 22:41:51 Expected and Actual scan responses are the same
2019/12/10 22:41:51 
--- Multi Scan 1 ---
2019/12/10 22:41:51 distinct = false
2019/12/10 22:41:51 Using n1ql client
2019/12/10 22:41:51 Using n1ql client
2019/12/10 22:41:51 len(scanResults) = 1 MultiScanCount = 1
2019/12/10 22:41:51 Expected and Actual scan responses are the same
2019/12/10 22:41:51 
--- Multi Scan 2 ---
2019/12/10 22:41:51 distinct = false
2019/12/10 22:41:51 Using n1ql client
2019/12/10 22:41:51 Using n1ql client
2019/12/10 22:41:51 len(scanResults) = 555 MultiScanCount = 555
2019/12/10 22:41:51 Expected and Actual scan responses are the same
2019/12/10 22:41:51 
--- Multi Scan 3 ---
2019/12/10 22:41:51 distinct = false
2019/12/10 22:41:52 Using n1ql client
2019/12/10 22:41:52 Using n1ql client
2019/12/10 22:41:52 len(scanResults) = 872 MultiScanCount = 872
2019/12/10 22:41:52 Expected and Actual scan responses are the same
2019/12/10 22:41:52 
--- Multi Scan 4 ---
2019/12/10 22:41:52 distinct = false
2019/12/10 22:41:53 Using n1ql client
2019/12/10 22:41:53 Using n1ql client
2019/12/10 22:41:53 len(scanResults) = 287 MultiScanCount = 287
2019/12/10 22:41:53 Expected and Actual scan responses are the same
2019/12/10 22:41:53 
--- Multi Scan 5 ---
2019/12/10 22:41:53 distinct = false
2019/12/10 22:41:53 Using n1ql client
2019/12/10 22:41:53 Using n1ql client
2019/12/10 22:41:53 len(scanResults) = 5254 MultiScanCount = 5254
2019/12/10 22:41:53 Expected and Actual scan responses are the same
2019/12/10 22:41:53 
--- Multi Scan 6 ---
2019/12/10 22:41:53 distinct = false
2019/12/10 22:41:54 Using n1ql client
2019/12/10 22:41:54 Using n1ql client
2019/12/10 22:41:54 len(scanResults) = 5566 MultiScanCount = 5566
2019/12/10 22:41:54 Expected and Actual scan responses are the same
2019/12/10 22:41:54 
--- Multi Scan 7 ---
2019/12/10 22:41:54 distinct = false
2019/12/10 22:41:55 Using n1ql client
2019/12/10 22:41:55 Using n1ql client
2019/12/10 22:41:55 len(scanResults) = 8 MultiScanCount = 8
2019/12/10 22:41:55 Expected and Actual scan responses are the same
2019/12/10 22:41:55 

--------- With DISTINCT True ---------
2019/12/10 22:41:55 
--- ScanAllNoFilter ---
2019/12/10 22:41:55 distinct = true
2019/12/10 22:41:55 Using n1ql client
2019/12/10 22:41:55 MultiScanCount = 999 ExpectedMultiScanCount = 999
2019/12/10 22:41:55 
--- ScanAllFilterNil ---
2019/12/10 22:41:55 distinct = true
2019/12/10 22:41:56 Using n1ql client
2019/12/10 22:41:56 MultiScanCount = 999 ExpectedMultiScanCount = 999
2019/12/10 22:41:56 
--- ScanAll_AllFiltersNil ---
2019/12/10 22:41:56 distinct = true
2019/12/10 22:41:56 Using n1ql client
2019/12/10 22:41:57 MultiScanCount = 999 ExpectedMultiScanCount = 999
2019/12/10 22:41:57 
--- SingleSeek ---
2019/12/10 22:41:57 distinct = true
2019/12/10 22:41:57 Using n1ql client
2019/12/10 22:41:57 MultiScanCount = 1 ExpectedMultiScanCount = 1
2019/12/10 22:41:57 
--- MultipleSeek ---
2019/12/10 22:41:57 distinct = true
2019/12/10 22:41:58 Using n1ql client
2019/12/10 22:41:58 MultiScanCount = 2 ExpectedMultiScanCount = 2
2019/12/10 22:41:58 
--- SimpleRange ---
2019/12/10 22:41:58 distinct = true
2019/12/10 22:41:58 Using n1ql client
2019/12/10 22:41:58 MultiScanCount = 227 ExpectedMultiScanCount = 227
2019/12/10 22:41:58 
--- NonOverlappingRanges ---
2019/12/10 22:41:58 distinct = true
2019/12/10 22:41:59 Using n1ql client
2019/12/10 22:41:59 MultiScanCount = 428 ExpectedMultiScanCount = 428
2019/12/10 22:41:59 
--- NonOverlappingFilters2 ---
2019/12/10 22:41:59 distinct = true
2019/12/10 22:41:59 Using n1ql client
2019/12/10 22:41:59 MultiScanCount = 1 ExpectedMultiScanCount = 1
2019/12/10 22:41:59 
--- OverlappingRanges ---
2019/12/10 22:41:59 distinct = true
2019/12/10 22:42:00 Using n1ql client
2019/12/10 22:42:00 MultiScanCount = 575 ExpectedMultiScanCount = 575
2019/12/10 22:42:00 
--- NonOverlappingFilters ---
2019/12/10 22:42:00 distinct = true
2019/12/10 22:42:00 Using n1ql client
2019/12/10 22:42:00 MultiScanCount = 186 ExpectedMultiScanCount = 186
2019/12/10 22:42:00 
--- OverlappingFilters ---
2019/12/10 22:42:00 distinct = true
2019/12/10 22:42:01 Using n1ql client
2019/12/10 22:42:01 MultiScanCount = 543 ExpectedMultiScanCount = 543
2019/12/10 22:42:01 
--- BoundaryFilters ---
2019/12/10 22:42:01 distinct = true
2019/12/10 22:42:01 Using n1ql client
2019/12/10 22:42:01 MultiScanCount = 172 ExpectedMultiScanCount = 172
2019/12/10 22:42:01 
--- SeekAndFilters_NonOverlapping ---
2019/12/10 22:42:01 distinct = true
2019/12/10 22:42:02 Using n1ql client
2019/12/10 22:42:02 MultiScanCount = 135 ExpectedMultiScanCount = 135
2019/12/10 22:42:02 
--- SeekAndFilters_Overlapping ---
2019/12/10 22:42:02 distinct = true
2019/12/10 22:42:02 Using n1ql client
2019/12/10 22:42:02 MultiScanCount = 134 ExpectedMultiScanCount = 134
2019/12/10 22:42:02 
--- SimpleRangeLowUnbounded ---
2019/12/10 22:42:02 distinct = false
2019/12/10 22:42:03 Using n1ql client
2019/12/10 22:42:03 MultiScanCount = 5618 ExpectedMultiScanCount = 5618
2019/12/10 22:42:03 
--- SimpleRangeHighUnbounded ---
2019/12/10 22:42:03 distinct = false
2019/12/10 22:42:03 Using n1ql client
2019/12/10 22:42:03 MultiScanCount = 3704 ExpectedMultiScanCount = 3704
2019/12/10 22:42:03 
--- SimpleRangeMultipleUnbounded ---
2019/12/10 22:42:03 distinct = false
2019/12/10 22:42:04 Using n1ql client
2019/12/10 22:42:04 MultiScanCount = 10002 ExpectedMultiScanCount = 10002
2019/12/10 22:42:04 
--- FiltersWithUnbounded ---
2019/12/10 22:42:04 distinct = false
2019/12/10 22:42:05 Using n1ql client
2019/12/10 22:42:05 MultiScanCount = 3173 ExpectedMultiScanCount = 3173
2019/12/10 22:42:05 
--- FiltersLowGreaterThanHigh ---
2019/12/10 22:42:05 distinct = false
2019/12/10 22:42:05 Using n1ql client
2019/12/10 22:42:05 MultiScanCount = 418 ExpectedMultiScanCount = 418
2019/12/10 22:42:05 

--------- Simple Index with 1 field ---------
2019/12/10 22:42:05 
--- SingleIndexSimpleRange ---
2019/12/10 22:42:05 distinct = true
2019/12/10 22:42:06 Using n1ql client
2019/12/10 22:42:06 MultiScanCount = 227 ExpectedMultiScanCount = 227
2019/12/10 22:42:06 
--- SingleIndex_SimpleRanges_NonOverlapping ---
2019/12/10 22:42:06 distinct = true
2019/12/10 22:42:06 Using n1ql client
2019/12/10 22:42:06 MultiScanCount = 713 ExpectedMultiScanCount = 713
2019/12/10 22:42:06 
--- SingleIndex_SimpleRanges_Overlapping ---
2019/12/10 22:42:06 distinct = true
2019/12/10 22:42:07 Using n1ql client
2019/12/10 22:42:07 MultiScanCount = 869 ExpectedMultiScanCount = 869
2019/12/10 22:42:07 

--------- Composite Index with 3 fields ---------
2019/12/10 22:42:07 
--- ScanAllNoFilter ---
2019/12/10 22:42:07 distinct = true
2019/12/10 22:42:07 Using n1ql client
2019/12/10 22:42:07 MultiScanCount = 999 ExpectedMultiScanCount = 999
2019/12/10 22:42:07 
--- ScanAllFilterNil ---
2019/12/10 22:42:07 distinct = true
2019/12/10 22:42:08 Using n1ql client
2019/12/10 22:42:08 MultiScanCount = 999 ExpectedMultiScanCount = 999
2019/12/10 22:42:08 
--- ScanAll_AllFiltersNil ---
2019/12/10 22:42:08 distinct = true
2019/12/10 22:42:09 Using n1ql client
2019/12/10 22:42:09 MultiScanCount = 999 ExpectedMultiScanCount = 999
2019/12/10 22:42:09 
--- 3FieldsSingleSeek ---
2019/12/10 22:42:09 distinct = true
2019/12/10 22:42:09 Using n1ql client
2019/12/10 22:42:09 MultiScanCount = 1 ExpectedMultiScanCount = 1
2019/12/10 22:42:09 
--- 3FieldsMultipleSeeks ---
2019/12/10 22:42:09 distinct = true
2019/12/10 22:42:10 Using n1ql client
2019/12/10 22:42:10 MultiScanCount = 3 ExpectedMultiScanCount = 3
2019/12/10 22:42:10 
--- 3FieldsMultipleSeeks_Identical ---
2019/12/10 22:42:10 distinct = true
2019/12/10 22:42:10 Using n1ql client
2019/12/10 22:42:10 MultiScanCount = 2 ExpectedMultiScanCount = 2
--- PASS: TestMultiScanDescCount (41.10s)
=== RUN   TestMultiScanDescOffset
2019/12/10 22:42:10 In SkipTestMultiScanDescOffset()
2019/12/10 22:42:10 

--------- Composite Index with 2 fields ---------
2019/12/10 22:42:10 
--- ScanAllNoFilter ---
2019/12/10 22:42:10 distinct = false
2019/12/10 22:42:11 Using n1ql client
2019/12/10 22:42:11 
--- ScanAllFilterNil ---
2019/12/10 22:42:11 distinct = false
2019/12/10 22:42:11 Using n1ql client
2019/12/10 22:42:12 
--- ScanAll_AllFiltersNil ---
2019/12/10 22:42:12 distinct = false
2019/12/10 22:42:12 Using n1ql client
2019/12/10 22:42:12 
--- SingleSeek ---
2019/12/10 22:42:12 distinct = false
2019/12/10 22:42:13 Using n1ql client
2019/12/10 22:42:13 
--- MultipleSeek ---
2019/12/10 22:42:13 distinct = false
2019/12/10 22:42:13 Using n1ql client
2019/12/10 22:42:13 
--- SimpleRange ---
2019/12/10 22:42:13 distinct = false
2019/12/10 22:42:14 Using n1ql client
2019/12/10 22:42:14 
--- NonOverlappingRanges ---
2019/12/10 22:42:14 distinct = false
2019/12/10 22:42:14 Using n1ql client
2019/12/10 22:42:14 
--- OverlappingRanges ---
2019/12/10 22:42:14 distinct = false
2019/12/10 22:42:15 Using n1ql client
2019/12/10 22:42:15 
--- NonOverlappingFilters ---
2019/12/10 22:42:15 distinct = false
2019/12/10 22:42:15 Using n1ql client
2019/12/10 22:42:15 
--- OverlappingFilters ---
2019/12/10 22:42:15 distinct = false
2019/12/10 22:42:16 Using n1ql client
2019/12/10 22:42:16 
--- BoundaryFilters ---
2019/12/10 22:42:16 distinct = false
2019/12/10 22:42:16 Using n1ql client
2019/12/10 22:42:16 
--- SeekAndFilters_NonOverlapping ---
2019/12/10 22:42:16 distinct = false
2019/12/10 22:42:17 Using n1ql client
2019/12/10 22:42:17 
--- SeekAndFilters_Overlapping ---
2019/12/10 22:42:17 distinct = false
2019/12/10 22:42:17 Using n1ql client
2019/12/10 22:42:18 
--- SimpleRangeLowUnbounded ---
2019/12/10 22:42:18 distinct = false
2019/12/10 22:42:18 Using n1ql client
2019/12/10 22:42:18 Expected and Actual scan responses are the same
2019/12/10 22:42:18 
--- SimpleRangeHighUnbounded ---
2019/12/10 22:42:18 distinct = false
2019/12/10 22:42:19 Using n1ql client
2019/12/10 22:42:19 Expected and Actual scan responses are the same
2019/12/10 22:42:19 
--- SimpleRangeMultipleUnbounded ---
2019/12/10 22:42:19 distinct = false
2019/12/10 22:42:19 Using n1ql client
2019/12/10 22:42:19 Expected and Actual scan responses are the same
2019/12/10 22:42:19 
--- FiltersWithUnbounded ---
2019/12/10 22:42:19 distinct = false
2019/12/10 22:42:20 Using n1ql client
2019/12/10 22:42:20 Expected and Actual scan responses are the same
2019/12/10 22:42:20 
--- FiltersLowGreaterThanHigh ---
2019/12/10 22:42:20 distinct = false
2019/12/10 22:42:20 Using n1ql client
2019/12/10 22:42:20 Expected and Actual scan responses are the same
2019/12/10 22:42:20 

--------- Simple Index with 1 field ---------
2019/12/10 22:42:20 
--- SingleIndexSimpleRange ---
2019/12/10 22:42:20 distinct = false
2019/12/10 22:42:21 Using n1ql client
2019/12/10 22:42:21 
--- SingleIndex_SimpleRanges_NonOverlapping ---
2019/12/10 22:42:21 distinct = false
2019/12/10 22:42:21 Using n1ql client
2019/12/10 22:42:21 
--- SingleIndex_SimpleRanges_Overlapping ---
2019/12/10 22:42:21 distinct = false
2019/12/10 22:42:22 Using n1ql client
2019/12/10 22:42:22 

--------- Composite Index with 3 fields ---------
2019/12/10 22:42:22 
--- ScanAllNoFilter ---
2019/12/10 22:42:22 distinct = false
2019/12/10 22:42:22 Using n1ql client
2019/12/10 22:42:23 
--- ScanAllFilterNil ---
2019/12/10 22:42:23 distinct = false
2019/12/10 22:42:23 Using n1ql client
2019/12/10 22:42:23 
--- ScanAll_AllFiltersNil ---
2019/12/10 22:42:23 distinct = false
2019/12/10 22:42:24 Using n1ql client
2019/12/10 22:42:24 
--- 3FieldsSingleSeek ---
2019/12/10 22:42:24 distinct = false
2019/12/10 22:42:24 Using n1ql client
2019/12/10 22:42:24 
--- 3FieldsMultipleSeeks ---
2019/12/10 22:42:24 distinct = false
2019/12/10 22:42:25 Using n1ql client
2019/12/10 22:42:25 
--- 3FieldsMultipleSeeks_Identical ---
2019/12/10 22:42:25 distinct = false
2019/12/10 22:42:25 Using n1ql client
--- PASS: TestMultiScanDescOffset (15.06s)
=== RUN   TestMultiScanDescDistinct
2019/12/10 22:42:25 In SkipTestMultiScanDescDistinct()
2019/12/10 22:42:25 

--------- Composite Index with 2 fields ---------
2019/12/10 22:42:25 
--- ScanAllNoFilter ---
2019/12/10 22:42:25 distinct = true
2019/12/10 22:42:26 Using n1ql client
2019/12/10 22:42:26 Expected and Actual scan responses are the same
2019/12/10 22:42:26 
--- ScanAllFilterNil ---
2019/12/10 22:42:26 distinct = true
2019/12/10 22:42:27 Using n1ql client
2019/12/10 22:42:27 Expected and Actual scan responses are the same
2019/12/10 22:42:27 
--- ScanAll_AllFiltersNil ---
2019/12/10 22:42:27 distinct = true
2019/12/10 22:42:27 Using n1ql client
2019/12/10 22:42:28 Expected and Actual scan responses are the same
2019/12/10 22:42:28 
--- SingleSeek ---
2019/12/10 22:42:28 distinct = true
2019/12/10 22:42:28 Using n1ql client
2019/12/10 22:42:28 Expected and Actual scan responses are the same
2019/12/10 22:42:28 
--- MultipleSeek ---
2019/12/10 22:42:28 distinct = true
2019/12/10 22:42:29 Using n1ql client
2019/12/10 22:42:29 Expected and Actual scan responses are the same
2019/12/10 22:42:29 
--- SimpleRange ---
2019/12/10 22:42:29 distinct = true
2019/12/10 22:42:29 Using n1ql client
2019/12/10 22:42:29 Expected and Actual scan responses are the same
2019/12/10 22:42:29 
--- NonOverlappingRanges ---
2019/12/10 22:42:29 distinct = true
2019/12/10 22:42:30 Using n1ql client
2019/12/10 22:42:30 Expected and Actual scan responses are the same
2019/12/10 22:42:30 
--- OverlappingRanges ---
2019/12/10 22:42:30 distinct = true
2019/12/10 22:42:30 Using n1ql client
2019/12/10 22:42:30 Expected and Actual scan responses are the same
2019/12/10 22:42:30 
--- NonOverlappingFilters ---
2019/12/10 22:42:30 distinct = true
2019/12/10 22:42:31 Using n1ql client
2019/12/10 22:42:31 Expected and Actual scan responses are the same
2019/12/10 22:42:31 
--- OverlappingFilters ---
2019/12/10 22:42:31 distinct = true
2019/12/10 22:42:31 Using n1ql client
2019/12/10 22:42:31 Expected and Actual scan responses are the same
2019/12/10 22:42:31 
--- BoundaryFilters ---
2019/12/10 22:42:31 distinct = true
2019/12/10 22:42:32 Using n1ql client
2019/12/10 22:42:32 Expected and Actual scan responses are the same
2019/12/10 22:42:32 
--- SeekAndFilters_NonOverlapping ---
2019/12/10 22:42:32 distinct = true
2019/12/10 22:42:32 Using n1ql client
2019/12/10 22:42:32 Expected and Actual scan responses are the same
2019/12/10 22:42:32 
--- SeekAndFilters_Overlapping ---
2019/12/10 22:42:32 distinct = true
2019/12/10 22:42:33 Using n1ql client
2019/12/10 22:42:33 Expected and Actual scan responses are the same
2019/12/10 22:42:33 
--- SimpleRangeLowUnbounded ---
2019/12/10 22:42:33 distinct = false
2019/12/10 22:42:33 Using n1ql client
2019/12/10 22:42:34 Expected and Actual scan responses are the same
2019/12/10 22:42:34 
--- SimpleRangeHighUnbounded ---
2019/12/10 22:42:34 distinct = false
2019/12/10 22:42:34 Using n1ql client
2019/12/10 22:42:34 Expected and Actual scan responses are the same
2019/12/10 22:42:34 
--- SimpleRangeMultipleUnbounded ---
2019/12/10 22:42:34 distinct = false
2019/12/10 22:42:35 Using n1ql client
2019/12/10 22:42:35 Expected and Actual scan responses are the same
2019/12/10 22:42:35 
--- FiltersWithUnbounded ---
2019/12/10 22:42:35 distinct = false
2019/12/10 22:42:35 Using n1ql client
2019/12/10 22:42:35 Expected and Actual scan responses are the same
2019/12/10 22:42:35 
--- FiltersLowGreaterThanHigh ---
2019/12/10 22:42:35 distinct = false
2019/12/10 22:42:36 Using n1ql client
2019/12/10 22:42:36 Expected and Actual scan responses are the same
2019/12/10 22:42:36 

--------- Simple Index with 1 field ---------
2019/12/10 22:42:36 
--- SingleIndexSimpleRange ---
2019/12/10 22:42:36 distinct = true
2019/12/10 22:42:36 Using n1ql client
2019/12/10 22:42:36 Expected and Actual scan responses are the same
2019/12/10 22:42:36 
--- SingleIndex_SimpleRanges_NonOverlapping ---
2019/12/10 22:42:36 distinct = true
2019/12/10 22:42:37 Using n1ql client
2019/12/10 22:42:37 Expected and Actual scan responses are the same
2019/12/10 22:42:37 
--- SingleIndex_SimpleRanges_Overlapping ---
2019/12/10 22:42:37 distinct = true
2019/12/10 22:42:37 Using n1ql client
2019/12/10 22:42:37 Expected and Actual scan responses are the same
2019/12/10 22:42:37 

--------- Composite Index with 3 fields ---------
2019/12/10 22:42:37 
--- ScanAllNoFilter ---
2019/12/10 22:42:37 distinct = true
2019/12/10 22:42:38 Using n1ql client
2019/12/10 22:42:38 Expected and Actual scan responses are the same
2019/12/10 22:42:38 
--- ScanAllFilterNil ---
2019/12/10 22:42:38 distinct = true
2019/12/10 22:42:39 Using n1ql client
2019/12/10 22:42:39 Expected and Actual scan responses are the same
2019/12/10 22:42:39 
--- ScanAll_AllFiltersNil ---
2019/12/10 22:42:39 distinct = true
2019/12/10 22:42:40 Using n1ql client
2019/12/10 22:42:40 Expected and Actual scan responses are the same
2019/12/10 22:42:40 
--- 3FieldsSingleSeek ---
2019/12/10 22:42:40 distinct = true
2019/12/10 22:42:40 Using n1ql client
2019/12/10 22:42:40 Expected and Actual scan responses are the same
2019/12/10 22:42:40 
--- 3FieldsMultipleSeeks ---
2019/12/10 22:42:40 distinct = true
2019/12/10 22:42:41 Using n1ql client
2019/12/10 22:42:41 Expected and Actual scan responses are the same
2019/12/10 22:42:41 
--- 3FieldsMultipleSeeks_Identical ---
2019/12/10 22:42:41 distinct = true
2019/12/10 22:42:41 Using n1ql client
2019/12/10 22:42:41 Expected and Actual scan responses are the same
--- PASS: TestMultiScanDescDistinct (15.86s)
=== RUN   TestGroupAggrSetup
2019/12/10 22:42:41 In TestGroupAggrSetup()
2019/12/10 22:42:41 Emptying the default bucket
2019/12/10 22:42:44 Flush Enabled on bucket default
2019/12/10 22:43:53 Flushed the bucket default
2019/12/10 22:43:58 Dropping the secondary index index_agg
2019/12/10 22:43:58 Populating the default bucket
2019/12/10 22:44:03 Created the secondary index index_agg. Waiting for it become active
2019/12/10 22:44:03 Index is now active
--- PASS: TestGroupAggrSetup (85.93s)
=== RUN   TestGroupAggrLeading
2019/12/10 22:44:07 In TestGroupAggrLeading()
2019/12/10 22:44:07 Total Scanresults = 7
2019/12/10 22:44:07 Expected and Actual scan responses are the same
2019/12/10 22:44:07 Total Scanresults = 3
2019/12/10 22:44:07 Expected and Actual scan responses are the same
--- PASS: TestGroupAggrLeading (0.01s)
=== RUN   TestGroupAggrNonLeading
2019/12/10 22:44:07 In TestGroupAggrNonLeading()
2019/12/10 22:44:07 Total Scanresults = 4
2019/12/10 22:44:07 Expected and Actual scan responses are the same
--- PASS: TestGroupAggrNonLeading (0.00s)
=== RUN   TestGroupAggrNoGroup
2019/12/10 22:44:07 In TestGroupAggrNoGroup()
2019/12/10 22:44:07 Total Scanresults = 1
2019/12/10 22:44:07 Expected and Actual scan responses are the same
--- PASS: TestGroupAggrNoGroup (0.00s)
=== RUN   TestGroupAggrMinMax
2019/12/10 22:44:07 In TestGroupAggrMinMax()
2019/12/10 22:44:07 Total Scanresults = 4
2019/12/10 22:44:07 Expected and Actual scan responses are the same
--- PASS: TestGroupAggrMinMax (0.01s)
=== RUN   TestGroupAggrMinMax2
2019/12/10 22:44:07 In TestGroupAggrMinMax()
2019/12/10 22:44:07 Total Scanresults = 1
2019/12/10 22:44:07 Expected and Actual scan responses are the same
--- PASS: TestGroupAggrMinMax2 (0.00s)
=== RUN   TestGroupAggrLeading_N1QLExprs
2019/12/10 22:44:07 In TestGroupAggrLeading_N1QLExprs()
2019/12/10 22:44:07 Total Scanresults = 17
2019/12/10 22:44:07 basicGroupAggrN1QLExprs1: Scan validation passed
2019/12/10 22:44:07 Total Scanresults = 9
2019/12/10 22:44:07 basicGroupAggrN1QLExprs2: Scan validation passed
--- PASS: TestGroupAggrLeading_N1QLExprs (0.15s)
=== RUN   TestGroupAggrLimit
2019/12/10 22:44:07 In TestGroupAggrLimit()
2019/12/10 22:44:07 Total Scanresults = 3
2019/12/10 22:44:07 Expected and Actual scan responses are the same
2019/12/10 22:44:07 Total Scanresults = 2
2019/12/10 22:44:07 Expected and Actual scan responses are the same
--- PASS: TestGroupAggrLimit (0.01s)
=== RUN   TestGroupAggrOffset
2019/12/10 22:44:07 In TestGroupAggrOffset()
2019/12/10 22:44:07 Total Scanresults = 3
2019/12/10 22:44:07 Expected and Actual scan responses are the same
2019/12/10 22:44:07 Total Scanresults = 2
2019/12/10 22:44:07 Expected and Actual scan responses are the same
--- PASS: TestGroupAggrOffset (0.01s)
=== RUN   TestGroupAggrCountN
2019/12/10 22:44:07 In TestGroupAggrCountN()
2019/12/10 22:44:07 Total Scanresults = 4
2019/12/10 22:44:07 Expected and Actual scan responses are the same
2019/12/10 22:44:07 Total Scanresults = 4
2019/12/10 22:44:07 Expected and Actual scan responses are the same
--- PASS: TestGroupAggrCountN (0.00s)
=== RUN   TestGroupAggrNoGroupNoMatch
2019/12/10 22:44:07 In TestGroupAggrNoGroupNoMatch()
2019/12/10 22:44:07 Total Scanresults = 1
2019/12/10 22:44:07 Expected and Actual scan responses are the same
--- PASS: TestGroupAggrNoGroupNoMatch (0.00s)
=== RUN   TestGroupAggrGroupNoMatch
2019/12/10 22:44:07 In TestGroupAggrGroupNoMatch()
2019/12/10 22:44:07 Total Scanresults = 0
2019/12/10 22:44:07 Expected and Actual scan responses are the same
--- PASS: TestGroupAggrGroupNoMatch (0.00s)
=== RUN   TestGroupAggrMultDataTypes
2019/12/10 22:44:07 In TestGroupAggrMultDataTypes()
2019/12/10 22:44:07 Total Scanresults = 8
2019/12/10 22:44:07 Expected and Actual scan responses are the same
--- PASS: TestGroupAggrMultDataTypes (0.00s)
=== RUN   TestGroupAggrDistinct
2019/12/10 22:44:07 In TestGroupAggrDistinct()
2019/12/10 22:44:07 Total Scanresults = 2
2019/12/10 22:44:07 Expected and Actual scan responses are the same
2019/12/10 22:44:07 Total Scanresults = 2
2019/12/10 22:44:07 Expected and Actual scan responses are the same
--- PASS: TestGroupAggrDistinct (0.00s)
=== RUN   TestGroupAggrDistinct2
2019/12/10 22:44:07 In TestGroupAggrDistinct2()
2019/12/10 22:44:07 Total Scanresults = 1
2019/12/10 22:44:07 Expected and Actual scan responses are the same
2019/12/10 22:44:07 Total Scanresults = 4
2019/12/10 22:44:07 Expected and Actual scan responses are the same
2019/12/10 22:44:07 Total Scanresults = 4
2019/12/10 22:44:07 Expected and Actual scan responses are the same
--- PASS: TestGroupAggrDistinct2 (0.01s)
=== RUN   TestGroupAggrNull
2019/12/10 22:44:07 In TestGroupAggrNull()
2019/12/10 22:44:07 Total Scanresults = 2
2019/12/10 22:44:07 Expected and Actual scan responses are the same
2019/12/10 22:44:07 Total Scanresults = 2
2019/12/10 22:44:07 Expected and Actual scan responses are the same
--- PASS: TestGroupAggrNull (0.00s)
=== RUN   TestGroupAggrInt64
2019/12/10 22:44:07 In TestGroupAggrInt64()
2019/12/10 22:44:07 Updating the default bucket
2019/12/10 22:44:08 Total Scanresults = 2
2019/12/10 22:44:08 Expected and Actual scan responses are the same
2019/12/10 22:44:08 Total Scanresults = 2
2019/12/10 22:44:08 Expected and Actual scan responses are the same
2019/12/10 22:44:08 Total Scanresults = 2
2019/12/10 22:44:08 Expected and Actual scan responses are the same
2019/12/10 22:44:08 Total Scanresults = 2
2019/12/10 22:44:08 Expected and Actual scan responses are the same
2019/12/10 22:44:08 Total Scanresults = 2
2019/12/10 22:44:08 Expected and Actual scan responses are the same
2019/12/10 22:44:08 Total Scanresults = 2
2019/12/10 22:44:08 Expected and Actual scan responses are the same
--- PASS: TestGroupAggrInt64 (0.40s)
=== RUN   TestGroupAggr1
2019/12/10 22:44:08 In TestGroupAggr1()
2019/12/10 22:44:08 In DropAllSecondaryIndexes()
2019/12/10 22:44:08 Index found:  index_company_name_age_desc
2019/12/10 22:44:08 Dropped index index_company_name_age_desc
2019/12/10 22:44:08 Index found:  index_agg
2019/12/10 22:44:08 Dropped index index_agg
2019/12/10 22:44:08 Index found:  #primary
2019/12/10 22:44:09 Dropped index #primary
2019/12/10 22:44:09 Index found:  index_companyname_desc
2019/12/10 22:44:09 Dropped index index_companyname_desc
2019/12/10 22:44:09 Index found:  index_company_desc
2019/12/10 22:44:09 Dropped index index_company_desc
2019/12/10 22:45:18 Flushed the bucket default
2019/12/10 22:45:27 Created the secondary index idx_aggrs. Waiting for it become active
2019/12/10 22:45:27 Index is now active
2019/12/10 22:45:27 Total Scanresults = 641
2019/12/10 22:45:28 Total Scanresults = 753
--- PASS: TestGroupAggr1 (80.13s)
=== RUN   TestGroupAggrArrayIndex
2019/12/10 22:45:28 In TestGroupAggrArrayIndex()
2019/12/10 22:45:33 Created the secondary index ga_arr1. Waiting for it become active
2019/12/10 22:45:33 Index is now active
2019/12/10 22:45:37 Created the secondary index ga_arr2. Waiting for it become active
2019/12/10 22:45:37 Index is now active
2019/12/10 22:45:37 Scenario 1
2019/12/10 22:45:37 Total Scanresults = 641
2019/12/10 22:45:38 Scenario 2
2019/12/10 22:45:38 Total Scanresults = 2835
2019/12/10 22:45:39 Scenario 3
2019/12/10 22:45:39 Total Scanresults = 1
2019/12/10 22:45:39 Scenario 4
2019/12/10 22:45:39 Total Scanresults = 991
2019/12/10 22:45:40 Scenario 5
2019/12/10 22:45:40 Total Scanresults = 2835
2019/12/10 22:45:41 Scenario 6
2019/12/10 22:45:41 Total Scanresults = 1
2019/12/10 22:45:41 Scenario 7
2019/12/10 22:45:41 Total Scanresults = 2931
2019/12/10 22:45:45 Scenario 8
2019/12/10 22:45:45 Total Scanresults = 1168
2019/12/10 22:45:46 Scenario 9
2019/12/10 22:45:46 Total Scanresults = 1
2019/12/10 22:45:46 Scenario 10
2019/12/10 22:45:46 Total Scanresults = 641
2019/12/10 22:45:46 Scenario 11
2019/12/10 22:45:46 Total Scanresults = 1168
2019/12/10 22:45:47 Scenario 12
2019/12/10 22:45:47 Total Scanresults = 1
2019/12/10 22:45:48 Scenario 13
2019/12/10 22:45:52 Total Scanresults = 1
2019/12/10 22:45:52 Count of scanResults is 1
2019/12/10 22:45:52 Value: [2 117]
--- PASS: TestGroupAggrArrayIndex (24.45s)
=== RUN   TestGroupAggr_FirstValidAggrOnly
2019/12/10 22:45:52 In TestGroupAggr_FirstValidAggrOnly()
2019/12/10 22:45:52 In DropAllSecondaryIndexes()
2019/12/10 22:45:52 Index found:  test_oneperprimarykey
2019/12/10 22:45:53 Dropped index test_oneperprimarykey
2019/12/10 22:45:53 Index found:  idx_aggrs
2019/12/10 22:45:53 Dropped index idx_aggrs
2019/12/10 22:45:53 Index found:  ga_arr1
2019/12/10 22:45:53 Dropped index ga_arr1
2019/12/10 22:45:53 Index found:  ga_arr2
2019/12/10 22:45:53 Dropped index ga_arr2
2019/12/10 22:45:53 Index found:  #primary
2019/12/10 22:45:54 Dropped index #primary
2019/12/10 22:46:07 Created the secondary index idx_asc_3field. Waiting for it become active
2019/12/10 22:46:07 Index is now active
2019/12/10 22:46:11 Created the secondary index idx_desc_3field. Waiting for it become active
2019/12/10 22:46:11 Index is now active
2019/12/10 22:46:11 === MIN no group by ===
2019/12/10 22:46:11 Total Scanresults = 1
2019/12/10 22:46:11 Count of scanResults is 1
2019/12/10 22:46:11 Value: ["ACCEL"]
2019/12/10 22:46:12 === MIN no group by, no row match ===
2019/12/10 22:46:12 Total Scanresults = 1
2019/12/10 22:46:12 Count of scanResults is 1
2019/12/10 22:46:12 Value: [null]
2019/12/10 22:46:12 === MIN with group by ===
2019/12/10 22:46:12 Total Scanresults = 641
2019/12/10 22:46:12 === MIN with group by, no row match ===
2019/12/10 22:46:12 Total Scanresults = 0
2019/12/10 22:46:13 === One Aggr, no group by ===
2019/12/10 22:46:13 Total Scanresults = 1
2019/12/10 22:46:13 Count of scanResults is 1
2019/12/10 22:46:13 Value: ["FANFARE"]
2019/12/10 22:46:13 === One Aggr, no group by, no row match ===
2019/12/10 22:46:13 Total Scanresults = 1
2019/12/10 22:46:13 Count of scanResults is 1
2019/12/10 22:46:13 Value: [null]
2019/12/10 22:46:13 === Multiple Aggr, no group by ===
2019/12/10 22:46:13 Total Scanresults = 1
2019/12/10 22:46:13 Count of scanResults is 1
2019/12/10 22:46:13 Value: ["FANFARE" 15]
2019/12/10 22:46:13 === Multiple Aggr, no group by, no row match ===
2019/12/10 22:46:13 Total Scanresults = 1
2019/12/10 22:46:13 Count of scanResults is 1
2019/12/10 22:46:13 Value: [null null]
2019/12/10 22:46:13 === No Aggr, 1 group by ===
2019/12/10 22:46:13 Total Scanresults = 211
2019/12/10 22:46:14 === Aggr on non-leading key, previous equality filter, no group ===
2019/12/10 22:46:14 Total Scanresults = 1
2019/12/10 22:46:14 Count of scanResults is 1
2019/12/10 22:46:14 Value: [59]
2019/12/10 22:46:14 === Aggr on non-leading key, previous equality filters, no group ===
2019/12/10 22:46:14 Total Scanresults = 1
2019/12/10 22:46:14 Count of scanResults is 1
2019/12/10 22:46:14 Value: [null]
2019/12/10 22:46:14 === Aggr on non-leading key, previous non-equality filters, no group ===
2019/12/10 22:46:14 Total Scanresults = 1
2019/12/10 22:46:14 Count of scanResults is 1
2019/12/10 22:46:14 Value: ["Adam"]
2019/12/10 22:46:14 === MIN on desc, no group ===
2019/12/10 22:46:14 Total Scanresults = 1
2019/12/10 22:46:14 Count of scanResults is 1
2019/12/10 22:46:14 Value: ["FANFARE"]
2019/12/10 22:46:14 === MAX on asc, no group ===
2019/12/10 22:46:14 Total Scanresults = 1
2019/12/10 22:46:14 Count of scanResults is 1
2019/12/10 22:46:14 Value: ["OZEAN"]
2019/12/10 22:46:14 === MAX on desc, no group ===
2019/12/10 22:46:14 Total Scanresults = 1
2019/12/10 22:46:14 Count of scanResults is 1
2019/12/10 22:46:14 Value: ["OZEAN"]
2019/12/10 22:46:15 === COUNT(DISTINCT const_expr, no group ===
2019/12/10 22:46:15 Total Scanresults = 1
2019/12/10 22:46:15 Count of scanResults is 1
2019/12/10 22:46:15 Value: [1]
2019/12/10 22:46:15 === COUNT(DISTINCT const_expr, no group, no row match ===
2019/12/10 22:46:15 Total Scanresults = 1
2019/12/10 22:46:15 Count of scanResults is 1
2019/12/10 22:46:15 Value: [0]
2019/12/10 22:46:15 === COUNT(const_expr, no group ===
2019/12/10 22:46:15 Total Scanresults = 1
2019/12/10 22:46:15 Count of scanResults is 1
2019/12/10 22:46:15 Value: [345]
2019/12/10 22:46:20 Created the secondary index indexMinAggr. Waiting for it become active
2019/12/10 22:46:20 Index is now active
2019/12/10 22:46:20 === Equality filter check: Equality for a, nil filter for b - inclusion 0, no filter for c ===
2019/12/10 22:46:20 Total Scanresults = 1
2019/12/10 22:46:20 Count of scanResults is 1
2019/12/10 22:46:20 Value: [5]
2019/12/10 22:46:20 === Equality filter check: Equality for a, nil filter for b - inclusion 3, no filter for c ===
2019/12/10 22:46:20 Total Scanresults = 1
2019/12/10 22:46:20 Count of scanResults is 1
2019/12/10 22:46:20 Value: [5]
2019/12/10 22:46:20 === Equality filter check: Equality for a, nil filter for b - inclusion 3, nil filter for c ===
2019/12/10 22:46:20 Total Scanresults = 1
2019/12/10 22:46:20 Count of scanResults is 1
2019/12/10 22:46:20 Value: [5]
--- PASS: TestGroupAggr_FirstValidAggrOnly (27.96s)
=== RUN   TestGroupAggrPrimary
2019/12/10 22:46:20 In TestGroupAggrPrimary()
2019/12/10 22:46:20 Total Scanresults = 1
2019/12/10 22:46:21 Total Scanresults = 1
2019/12/10 22:46:21 Total Scanresults = 1
2019/12/10 22:46:21 Total Scanresults = 1002
2019/12/10 22:46:21 Total Scanresults = 1002
2019/12/10 22:46:22 Total Scanresults = 1
2019/12/10 22:46:22 Total Scanresults = 1
2019/12/10 22:46:22 Total Scanresults = 1
2019/12/10 22:46:22 Total Scanresults = 1
2019/12/10 22:46:22 Total Scanresults = 1
2019/12/10 22:46:22 Total Scanresults = 1
2019/12/10 22:46:22 --- MB-28305 Scenario 1 ---
2019/12/10 22:46:22 Total Scanresults = 1
2019/12/10 22:46:22 Count of scanResults is 1
2019/12/10 22:46:22 Value: [0]
2019/12/10 22:46:22 --- MB-28305 Scenario 2 ---
2019/12/10 22:46:22 Total Scanresults = 1
2019/12/10 22:46:22 Count of scanResults is 1
2019/12/10 22:46:22 Value: [0]
2019/12/10 22:46:22 --- MB-28305 Scenario 3 ---
2019/12/10 22:46:22 Total Scanresults = 1
2019/12/10 22:46:22 Count of scanResults is 1
2019/12/10 22:46:22 Value: [0]
--- PASS: TestGroupAggrPrimary (1.49s)
=== RUN   TestIndexNodeRebalanceIn
2019/12/10 22:46:22 In TestIndexNodeRebalanceIn()
2019/12/10 22:46:22 This test will rebalance in an indexer node into the cluster
2019/12/10 22:46:22 Adding node: http://127.0.0.1:9002 with role: index to the cluster
2019/12/10 22:46:26 addNode: Successfully added node with hostname: 127.0.0.1:9002, res: {"otpNode":"n_2@127.0.0.1"}
2019-12-10T22:46:28.701+05:30 [Error] WatcherServer.runOnce() error : dial tcp 127.0.0.1:9112: connect: connection refused
2019-12-10T22:46:28.708+05:30 [Error] WatcherServer.runOnce() error : dial tcp 127.0.0.1:9112: connect: connection refused
2019-12-10T22:46:29.750+05:30 [Error] updateIndexerList(): Cannot locate cluster node.
2019-12-10T22:46:29.760+05:30 [Error] updateIndexerList(): Cannot locate cluster node.
2019/12/10 22:46:32 Rebalance progress: 100
--- PASS: TestIndexNodeRebalanceIn (9.71s)
=== RUN   TestIndexNodeRebalanceOut
2019/12/10 22:46:32 In TestIndexNodeRebalanceOut()
2019/12/10 22:46:32 This test will rebalance out an indexer node (127.0.0.1:9001) out of the cluster
2019/12/10 22:46:32 Removing node(s): [127.0.0.1:9001] from the cluster
2019/12/10 22:46:37 Rebalance progress: 0
2019/12/10 22:46:42 Rebalance progress: 1.34548611111111
2019/12/10 22:46:47 Rebalance progress: 2.864583333333333
2019/12/10 22:46:52 Rebalance progress: 4.427083333333333
2019/12/10 22:46:57 Rebalance progress: 5.902777777777778
2019/12/10 22:47:02 Rebalance progress: 7.378472222222222
2019/12/10 22:47:07 Rebalance progress: 9.114583333333334
2019/12/10 22:47:12 Rebalance progress: 10.67708333333333
2019/12/10 22:47:17 Rebalance progress: 12.19618055555556
2019/12/10 22:47:22 Rebalance progress: 13.54166666666667
2019/12/10 22:47:27 Rebalance progress: 15.19097222222222
2019/12/10 22:47:32 Rebalance progress: 16.71006944444444
2019/12/10 22:47:37 Rebalance progress: 18.18576388888889
2019/12/10 22:47:42 Rebalance progress: 19.87847222222222
2019/12/10 22:47:47 Rebalance progress: 21.35416666666667
2019/12/10 22:47:52 Rebalance progress: 23.37239583333333
2019/12/10 22:47:57 Rebalance progress: 25.36892361111111
2019/12/10 22:48:02 Rebalance progress: 27.34375
2019/12/10 22:48:07 Rebalance progress: 29.03645833333333
2019/12/10 22:48:12 Rebalance progress: 30.72916666666667
2019/12/10 22:48:17 Rebalance progress: 32.09635416666666
2019/12/10 22:48:22 Rebalance progress: 33.33333333333334
2019/12/10 22:48:27 Rebalance progress: 33.33333333333334
2019/12/10 22:48:32 Rebalance progress: 38.33333333333333
2019/12/10 22:48:37 Rebalance progress: 70.83333333333333
2019/12/10 22:48:42 Rebalance progress: 82.83333333333333
2019-12-10T22:48:42.859+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:9106.  Error = EOF. Kill Pipe.
2019-12-10T22:48:42.859+05:30 [Error] WatcherServer.runOnce() : Watcher terminated unexpectedly.
2019-12-10T22:48:42.859+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:9106.  Error = EOF. Kill Pipe.
2019-12-10T22:48:42.860+05:30 [Error] WatcherServer.runOnce() : Watcher terminated unexpectedly.
2019/12/10 22:48:47 Rebalance failed. See logs for detailed reason. You can try again.
--- FAIL: TestIndexNodeRebalanceOut (135.25s)
    cluster_setup.go:24: Error during rebalance, err: Rebalance failed
=== RUN   TestFailoverAndRebalance
2019/12/10 22:48:47 In TestFailoverAndRebalance()
2019/12/10 22:48:47 This test will failover and rebalance-out an indexer node (127.0.0.1:9002) out of the cluster
--- FAIL: TestFailoverAndRebalance (0.02s)
    setm_rebalance_test.go:87: Unexpected cluster status: map[172.31.5.112:9000:[kv n1ql] 127.0.0.1:9001:[index kv] 127.0.0.1:9002:[index]]
=== RUN   TestResetCluster
2019/12/10 22:48:47 Removing node(s): [127.0.0.1:9001 127.0.0.1:9002 127.0.0.1:9003] from the cluster
2019-12-10T22:48:50.063+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:9112.  Error = read tcp 127.0.0.1:51196->127.0.0.1:9112: use of closed network connection. Kill Pipe.
2019-12-10T22:48:50.074+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:9112.  Error = read tcp 127.0.0.1:51194->127.0.0.1:9112: use of closed network connection. Kill Pipe.
2019/12/10 22:48:52 Rebalance progress: 100
2019/12/10 22:48:55 Adding node: http://127.0.0.1:9001 with role: index to the cluster
2019/12/10 22:48:59 addNode: Successfully added node with hostname: 127.0.0.1:9001, res: {"otpNode":"n_1@127.0.0.1"}
2019-12-10T22:49:03.535+05:30 [Error] WatcherServer.runOnce() error : dial tcp 127.0.0.1:9106: connect: connection refused
2019-12-10T22:49:03.540+05:30 [Error] WatcherServer.runOnce() error : dial tcp 127.0.0.1:9106: connect: connection refused
2019/12/10 22:49:04 Rebalance progress: 25
2019-12-10T22:49:04.605+05:30 [Error] updateIndexerList(): Cannot locate cluster node.
2019/12/10 22:49:09 Rebalance progress: 100
--- PASS: TestResetCluster (21.69s)
=== RUN   TestAlterIndexIncrReplica
2019/12/10 22:49:09 In TestAlterIndexIncrReplica()
2019/12/10 22:49:09 This test create an index with one replica and then increments replica count to 2
2019/12/10 22:49:09 Removing node(s): [127.0.0.1:9001 127.0.0.1:9002 127.0.0.1:9003] from the cluster
2019-12-10T22:49:10.261+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:9106.  Error = EOF. Kill Pipe.
2019-12-10T22:49:10.261+05:30 [Error] WatcherServer.runOnce() : Watcher terminated unexpectedly.
2019-12-10T22:49:10.267+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:9106.  Error = EOF. Kill Pipe.
2019-12-10T22:49:10.267+05:30 [Error] WatcherServer.runOnce() : Watcher terminated unexpectedly.
2019/12/10 22:49:14 Rebalance progress: 100
2019/12/10 22:49:17 Adding node: http://127.0.0.1:9001 with role: index to the cluster
2019/12/10 22:49:20 addNode: Successfully added node with hostname: 127.0.0.1:9001, res: {"otpNode":"n_1@127.0.0.1"}
2019-12-10T22:49:25.163+05:30 [Error] WatcherServer.runOnce() error : dial tcp 127.0.0.1:9106: connect: connection refused
2019-12-10T22:49:25.168+05:30 [Error] WatcherServer.runOnce() error : dial tcp 127.0.0.1:9106: connect: connection refused
2019/12/10 22:49:25 Rebalance progress: 25
2019-12-10T22:49:26.222+05:30 [Error] updateIndexerList(): Cannot locate cluster node.
2019-12-10T22:49:26.228+05:30 [Error] updateIndexerList(): Cannot locate cluster node.
2019/12/10 22:49:30 Rebalance progress: 100
2019/12/10 22:49:30 Adding node: http://127.0.0.1:9002 with role: index to the cluster
2019/12/10 22:49:34 addNode: Successfully added node with hostname: 127.0.0.1:9002, res: {"otpNode":"n_2@127.0.0.1"}
2019-12-10T22:49:36.114+05:30 [Error] WatcherServer.runOnce() error : dial tcp 127.0.0.1:9112: connect: connection refused
2019-12-10T22:49:36.137+05:30 [Error] WatcherServer.runOnce() error : dial tcp 127.0.0.1:9112: connect: connection refused
2019-12-10T22:49:37.297+05:30 [Error] updateIndexerList(): Cannot locate cluster node.
2019/12/10 22:49:39 Rebalance progress: 100
2019/12/10 22:49:39 Adding node: http://127.0.0.1:9003 with role: index to the cluster
2019/12/10 22:49:44 addNode: Successfully added node with hostname: 127.0.0.1:9003, res: {"otpNode":"n_3@127.0.0.1"}
2019-12-10T22:49:46.445+05:30 [Error] WatcherServer.runOnce() error : dial tcp 127.0.0.1:9118: connect: connection refused
2019-12-10T22:49:46.458+05:30 [Error] WatcherServer.runOnce() error : dial tcp 127.0.0.1:9118: connect: connection refused
2019-12-10T22:49:47.794+05:30 [Error] updateIndexerList(): Cannot locate cluster node.
2019-12-10T22:49:47.803+05:30 [Error] updateIndexerList(): Cannot locate cluster node.
2019/12/10 22:49:49 Rebalance progress: 100
2019/12/10 22:50:54 Flushed the bucket default
2019/12/10 22:51:01 Created the secondary index idx_1. Waiting for it become active
2019/12/10 22:51:01 Index is now active
2019/12/10 22:51:01 Executing alter index command: alter index `default`.idx_1 with {"action":"replica_count", "num_replica":2}
2019/12/10 22:51:11 Using n1ql client
2019/12/10 22:51:11 Using n1ql client
2019/12/10 22:51:11 Using n1ql client
2019/12/10 22:51:11 Using n1ql client
2019/12/10 22:51:11 Using n1ql client
2019/12/10 22:51:11 Using n1ql client
2019/12/10 22:51:11 Using n1ql client
2019/12/10 22:51:11 Using n1ql client
2019/12/10 22:51:11 Using n1ql client
2019/12/10 22:51:11 Using n1ql client
2019/12/10 22:51:11 Using n1ql client
2019/12/10 22:51:11 Using n1ql client
2019/12/10 22:51:11 Using n1ql client
2019/12/10 22:51:11 Using n1ql client
2019/12/10 22:51:11 Using n1ql client
2019/12/10 22:51:11 Using n1ql client
2019/12/10 22:51:11 Using n1ql client
2019/12/10 22:51:11 Using n1ql client
2019/12/10 22:51:11 Using n1ql client
2019/12/10 22:51:11 Using n1ql client
2019/12/10 22:51:11 Using n1ql client
2019/12/10 22:51:11 Using n1ql client
2019/12/10 22:51:11 Using n1ql client
2019/12/10 22:51:11 Using n1ql client
2019/12/10 22:51:11 Using n1ql client
2019/12/10 22:51:11 Using n1ql client
2019/12/10 22:51:11 Using n1ql client
2019/12/10 22:51:11 Using n1ql client
2019/12/10 22:51:11 Using n1ql client
2019/12/10 22:51:11 Using n1ql client
2019/12/10 22:51:11 Using n1ql client
2019/12/10 22:51:11 Using n1ql client
2019/12/10 22:51:11 Using n1ql client
2019/12/10 22:51:11 Using n1ql client
2019/12/10 22:51:11 Using n1ql client
2019/12/10 22:51:11 Using n1ql client
2019/12/10 22:51:11 Using n1ql client
2019/12/10 22:51:11 Using n1ql client
2019/12/10 22:51:11 Using n1ql client
2019/12/10 22:51:11 Using n1ql client
2019/12/10 22:51:11 Using n1ql client
2019/12/10 22:51:11 Using n1ql client
2019/12/10 22:51:11 Using n1ql client
2019/12/10 22:51:11 Using n1ql client
2019/12/10 22:51:11 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
2019/12/10 22:51:12 Using n1ql client
--- PASS: TestAlterIndexIncrReplica (123.12s)
=== RUN   TestResetCluster_1
2019/12/10 22:51:12 Removing node(s): [127.0.0.1:9001 127.0.0.1:9002 127.0.0.1:9003] from the cluster
2019-12-10T22:51:14.632+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:9106.  Error = EOF. Kill Pipe.
2019-12-10T22:51:14.632+05:30 [Error] WatcherServer.runOnce() : Watcher terminated unexpectedly.
2019-12-10T22:51:14.633+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:9106.  Error = EOF. Kill Pipe.
2019-12-10T22:51:14.633+05:30 [Error] WatcherServer.runOnce() : Watcher terminated unexpectedly.
2019-12-10T22:51:14.905+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:9112.  Error = EOF. Kill Pipe.
2019-12-10T22:51:14.905+05:30 [Error] WatcherServer.runOnce() : Watcher terminated unexpectedly.
2019-12-10T22:51:14.907+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:9112.  Error = EOF. Kill Pipe.
2019-12-10T22:51:14.907+05:30 [Error] WatcherServer.runOnce() : Watcher terminated unexpectedly.
2019-12-10T22:51:15.193+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:9118.  Error = read tcp 127.0.0.1:36514->127.0.0.1:9118: use of closed network connection. Kill Pipe.
2019-12-10T22:51:15.202+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:9118.  Error = read tcp 127.0.0.1:36516->127.0.0.1:9118: use of closed network connection. Kill Pipe.
2019/12/10 22:51:17 Rebalance progress: 100
2019/12/10 22:51:20 Adding node: http://127.0.0.1:9001 with role: index to the cluster
2019/12/10 22:51:24 addNode: Successfully added node with hostname: 127.0.0.1:9001, res: {"otpNode":"n_1@127.0.0.1"}
2019-12-10T22:51:28.731+05:30 [Error] WatcherServer.runOnce() error : dial tcp 127.0.0.1:9106: connect: connection refused
2019-12-10T22:51:28.746+05:30 [Error] WatcherServer.runOnce() error : dial tcp 127.0.0.1:9106: connect: connection refused
2019/12/10 22:51:29 Rebalance progress: 75
2019-12-10T22:51:29.798+05:30 [Error] updateIndexerList(): Cannot locate cluster node.
2019/12/10 22:51:34 Rebalance progress: 100
--- PASS: TestResetCluster_1 (22.24s)
=== RUN   TestIdxCorruptBasicSanityMultipleIndices
2019/12/10 22:51:34 In DropAllSecondaryIndexes()
Creating two indices ...
2019/12/10 22:51:47 Created the secondary index corrupt_idx1_age. Waiting for it become active
2019/12/10 22:51:47 Index is now active
2019/12/10 22:51:51 Created the secondary index corrupt_idx2_company. Waiting for it become active
2019/12/10 22:51:51 Index is now active
hosts = [127.0.0.1:9108]
2019/12/10 22:51:51 Corrupting index corrupt_idx1_age slicePath /opt/build/ns_server/data/n_1/data/@2i/default_corrupt_idx1_age_288163485213536601_0.index
2019/12/10 22:51:51 Corrupting index corrupt_idx1_age slicePath /opt/build/ns_server/data/n_1/data/@2i/default_corrupt_idx1_age_288163485213536601_0.index filepath /opt/build/ns_server/data/n_1/data/@2i/default_corrupt_idx1_age_288163485213536601_0.index/data.fdb.0
2019/12/10 22:51:51 Written data to offset  126975
2019/12/10 22:51:51 FDBCorrupt: Corrupted header for block id 30
2019/12/10 22:51:51 Written data to offset  53247
2019/12/10 22:51:51 FDBCorrupt: Corrupted header for block id 12
2019/12/10 22:51:51 Written data to offset  45055
2019/12/10 22:51:51 FDBCorrupt: Corrupted header for block id 10
Restarting indexer process ...
2019/12/10 22:51:51 []
2019-12-10T22:51:51.141+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:9106.  Error = EOF. Kill Pipe.
2019-12-10T22:51:51.141+05:30 [Error] WatcherServer.runOnce() : Watcher terminated unexpectedly.
2019-12-10T22:51:51.141+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:9106.  Error = EOF. Kill Pipe.
2019-12-10T22:51:51.141+05:30 [Error] WatcherServer.runOnce() : Watcher terminated unexpectedly.
2019/12/10 22:52:22 Using n1ql client
2019-12-10T22:52:22.122+05:30 [Error] transport error between 127.0.0.1:44538->127.0.0.1:9107: write tcp 127.0.0.1:44538->127.0.0.1:9107: write: broken pipe
2019-12-10T22:52:22.122+05:30 [Error] [GsiScanClient:"127.0.0.1:9107"] 2397663244430597608 request transport failed `write tcp 127.0.0.1:44538->127.0.0.1:9107: write: broken pipe`
2019-12-10T22:52:22.122+05:30 [Error] PickRandom: Fail to find indexer for all index partitions. Num partition 1.  Partition with instances 0 
2019/12/10 22:52:22 Using n1ql client
--- PASS: TestIdxCorruptBasicSanityMultipleIndices (47.68s)
=== RUN   TestIdxCorruptPartitionedIndex
Not running TestPartitionedIndex for forestdb
--- PASS: TestIdxCorruptPartitionedIndex (0.00s)
=== RUN   TestIdxCorruptMOITwoSnapsOneCorrupt
Not running TestMOITwoSnapsOneCorrupt for forestdb
--- PASS: TestIdxCorruptMOITwoSnapsOneCorrupt (0.00s)
=== RUN   TestIdxCorruptMOITwoSnapsBothCorrupt
Not running TestMOITwoSnapsBothCorrupt for forestdb
--- PASS: TestIdxCorruptMOITwoSnapsBothCorrupt (0.00s)
=== RUN   TestIdxCorruptBackup
2019/12/10 22:52:22 DBG: ChangeIndexerSettings: nodes = [0xc00c07fd80]
2019/12/10 22:52:22 DBG: ChangeIndexerSettings: adminurl = 127.0.0.1:9106 host 127.0.0.1 sport 9106
2019/12/10 22:52:22 Changing config key indexer.settings.enable_corrupt_index_backup to value true
Creating index ...
2019-12-10T22:52:22.279+05:30 [Info] Planner::processCreateToken: There is no create token to process for node 127.0.0.1:9001
2019-12-10T22:52:22.304+05:30 [Info] Planner::processDeleteToken: There is no delete token to process for node 127.0.0.1:9001
2019-12-10T22:52:22.330+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node 127.0.0.1:9001
2019-12-10T22:52:22.408+05:30 [Info] ************ Indexer Layout *************
2019-12-10T22:52:22.408+05:30 [Info] Score: 0
2019-12-10T22:52:22.408+05:30 [Info] variation: 0
2019-12-10T22:52:22.408+05:30 [Info] ElapsedTime: 7ms
2019-12-10T22:52:22.408+05:30 [Info] ConvergenceTime: 2us
2019-12-10T22:52:22.408+05:30 [Info] Iteration: 0
2019-12-10T22:52:22.408+05:30 [Info] Move: 0
2019-12-10T22:52:22.408+05:30 [Info] ----------------------------------------
2019-12-10T22:52:22.408+05:30 [Info] Indexer Memory Mean 1258291200 (1.17188G)
2019-12-10T22:52:22.408+05:30 [Info] Indexer Memory Deviation 0 (0) (0.00%)
2019-12-10T22:52:22.408+05:30 [Info] Indexer Memory Utilization 0.8000
2019-12-10T22:52:22.408+05:30 [Info] Indexer CPU Mean 0.0000
2019-12-10T22:52:22.408+05:30 [Info] Indexer CPU Deviation 0.00 (0.00%)
2019-12-10T22:52:22.408+05:30 [Info] Indexer CPU Utilization 0.0000
2019-12-10T22:52:22.408+05:30 [Info] Indexer IO Mean 0.0000
2019-12-10T22:52:22.408+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2019-12-10T22:52:22.408+05:30 [Info] Indexer Drain Rate Mean 0.0000
2019-12-10T22:52:22.408+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2019-12-10T22:52:22.408+05:30 [Info] Indexer Scan Rate Mean 0.0000
2019-12-10T22:52:22.408+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2019-12-10T22:52:22.408+05:30 [Info] Indexer Data Size Mean 1258350150 (1.17193G)
2019-12-10T22:52:22.408+05:30 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2019-12-10T22:52:22.408+05:30 [Info] Total Index Data (from non-deleted node) 0
2019-12-10T22:52:22.408+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2019-12-10T22:52:22.408+05:30 [Info] No. Index (from non-deleted node) 0
2019-12-10T22:52:22.408+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2019-12-10T22:52:22.408+05:30 [Info] ----------------------------------------
2019-12-10T22:52:22.408+05:30 [Info] Number of indexes: 2
2019-12-10T22:52:22.408+05:30 [Info] Number of indexers: 1
2019-12-10T22:52:22.408+05:30 [Info] Avg Index Size: 629145600 (600M)
2019-12-10T22:52:22.408+05:30 [Info] Max Index Size: 1258291200 (1.17188G)
2019-12-10T22:52:22.408+05:30 [Info] Max Indexer Overhead: 0 (0)
2019-12-10T22:52:22.408+05:30 [Info] Avg Index Cpu: 0.0000
2019-12-10T22:52:22.408+05:30 [Info] Max Index Cpu: 0.0000
2019-12-10T22:52:22.408+05:30 [Info] Num Estimation Run: 1
2019-12-10T22:52:22.408+05:30 [Info] ----------------------------------------
2019-12-10T22:52:22.408+05:30 [Info] Memory Quota 1572864000 (1.46484G)
2019-12-10T22:52:22.408+05:30 [Info] CPU Quota 4
2019-12-10T22:52:22.408+05:30 [Info] Max Cpu Utilization -1
2019-12-10T22:52:22.408+05:30 [Info] Max Memory Utilization -1
2019-12-10T22:52:22.408+05:30 [Info] ----------------------------------------
2019-12-10T22:52:22.408+05:30 [Info] Total iteration: 0
2019-12-10T22:52:22.408+05:30 [Info] RandomSwap time: 0ns
2019-12-10T22:52:22.408+05:30 [Info] RandomSwap call: 0
2019-12-10T22:52:22.408+05:30 [Info] RandomSwap iteration: 0
2019-12-10T22:52:22.408+05:30 [Info] RandomMove time: 0ns
2019-12-10T22:52:22.408+05:30 [Info] RandomMove call: 0 (empty index 0)
2019-12-10T22:52:22.408+05:30 [Info] ExhaustMove time: 0ns
2019-12-10T22:52:22.408+05:30 [Info] ExhaustMove call: 0
2019-12-10T22:52:22.408+05:30 [Info] ExhaustSwap time: 0ns
2019-12-10T22:52:22.408+05:30 [Info] ExhaustSwap call: 0
2019-12-10T22:52:22.408+05:30 [Info] ----------------------------------------
2019-12-10T22:52:22.408+05:30 [Info] 
2019-12-10T22:52:22.408+05:30 [Info] Indexer serverGroup:Group 1, nodeId:127.0.0.1:9001, nodeUUID:f7d69a0ebb5dfec6098045fb31c56ad5, useLiveData:true
2019-12-10T22:52:22.408+05:30 [Info] Indexer total memory:1258291200 (1.17188G), mem:1258291200 (1.17188G), overhead:0 (0), min:1258302990 (1.17189G), data:1258350150 (1.17193G) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:2
2019-12-10T22:52:22.408+05:30 [Info] Indexer isDeleted:false isNew:false exclude: meetConstraint:true
2019-12-10T22:52:22.408+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:52:22.408+05:30 [Info] 		Index name:corrupt_idx2_company 0, bucket:default, defnId:6337225028508673115, instId:11699873349970432747, Partition: 0, new/moved:false
2019-12-10T22:52:22.409+05:30 [Info] 		Index total memory:0 (0), mem:0 (0), overhead:0 (0), min:11790 (11.5137K), data:58950 (57.5684K) cpu:0.0000 io:0 (0) scan:0 drain:0
2019-12-10T22:52:22.409+05:30 [Info] 		Index resident:0% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:52:22.409+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:52:22.409+05:30 [Info] 		Index name:corrupt_idx6_age 0, bucket:default, defnId:2943406376927672045, instId:5188740215458289680, Partition: 0, new/moved:true
2019-12-10T22:52:22.409+05:30 [Info] 		Index total memory:1258291200 (1.17188G), mem:1258291200 (1.17188G), overhead:0 (0), min:1258291200 (1.17188G), data:1258291200 (1.17188G) cpu:0.0000 io:0 (0) scan:0 drain:0
2019-12-10T22:52:22.409+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:52:22.409+05:30 [Info] ****************************************
2019-12-10T22:52:22.409+05:30 [Info] send commit create request to watcher 127.0.0.1:9106
2019-12-10T22:52:22.499+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:52:22.499+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:52:22.503+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:52:22.503+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:52:22.504+05:30 [Info] switched currmeta from 1622 -> 1624 force true 
2019-12-10T22:52:22.508+05:30 [Info] switched currmeta from 1626 -> 1628 force true 
2019-12-10T22:52:22.600+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:52:22.600+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:52:22.603+05:30 [Info] switched currmeta from 1628 -> 1630 force true 
2019-12-10T22:52:22.606+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:52:22.606+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:52:22.609+05:30 [Info] switched currmeta from 1624 -> 1626 force true 
2019-12-10T22:52:22.641+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:52:22.641+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:52:22.648+05:30 [Info] switched currmeta from 1630 -> 1630 force true 
2019-12-10T22:52:22.655+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:52:22.655+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:52:22.659+05:30 [Info] switched currmeta from 1626 -> 1626 force true 
2019-12-10T22:52:22.747+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:52:22.747+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:52:22.761+05:30 [Info] switched currmeta from 1626 -> 1627 force true 
2019-12-10T22:52:22.771+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:52:22.771+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:52:22.782+05:30 [Info] switched currmeta from 1630 -> 1631 force true 
2019-12-10T22:52:24.962+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:52:24.962+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:52:24.964+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:52:24.964+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:52:24.965+05:30 [Info] switched currmeta from 1627 -> 1628 force true 
2019-12-10T22:52:24.970+05:30 [Info] switched currmeta from 1631 -> 1632 force true 
2019-12-10T22:52:25.810+05:30 [Info] CreateIndex 2943406376927672045 default/corrupt_idx6_age using:forestdb exprType:N1QL whereExpr:() secExprs:([`age`]) desc:[false] isPrimary:false scheme:SINGLE  partitionKeys:([]) with: - elapsed(3.655086596s) err()
2019/12/10 22:52:25 Created the secondary index corrupt_idx6_age. Waiting for it become active
2019-12-10T22:52:25.810+05:30 [Info] metadata provider version changed 1632 -> 1633
2019-12-10T22:52:25.810+05:30 [Info] switched currmeta from 1632 -> 1633 force false 
2019/12/10 22:52:25 Index is now active
2019-12-10T22:52:25.920+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:52:25.920+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:52:25.936+05:30 [Info] switched currmeta from 1628 -> 1629 force true 
hosts = [127.0.0.1:9108]
2019/12/10 22:52:25 Corrupting index corrupt_idx6_age slicePath /opt/build/ns_server/data/n_1/data/@2i/default_corrupt_idx6_age_17833433843410310327_0.index
2019/12/10 22:52:25 Corrupting index corrupt_idx6_age slicePath /opt/build/ns_server/data/n_1/data/@2i/default_corrupt_idx6_age_17833433843410310327_0.index filepath /opt/build/ns_server/data/n_1/data/@2i/default_corrupt_idx6_age_17833433843410310327_0.index/data.fdb.0
2019/12/10 22:52:25 Written data to offset  126975
2019/12/10 22:52:25 FDBCorrupt: Corrupted header for block id 30
2019/12/10 22:52:25 Written data to offset  53247
2019/12/10 22:52:25 FDBCorrupt: Corrupted header for block id 12
2019/12/10 22:52:25 Written data to offset  45055
2019/12/10 22:52:25 FDBCorrupt: Corrupted header for block id 10
Restarting indexer process ...
2019-12-10T22:52:25.957+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:52:25.957+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:52:25.964+05:30 [Info] switched currmeta from 1633 -> 1633 force true 
2019/12/10 22:52:26 []
2019-12-10T22:52:26.021+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:9106.  Error = EOF. Kill Pipe.
2019-12-10T22:52:26.021+05:30 [Error] WatcherServer.runOnce() : Watcher terminated unexpectedly.
2019-12-10T22:52:26.022+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:9106.  Error = EOF. Kill Pipe.
2019-12-10T22:52:26.023+05:30 [Error] WatcherServer.runOnce() : Watcher terminated unexpectedly.
2019-12-10T22:52:26.634+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:52:26.672+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:52:26.672+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:52:26.675+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:52:26.675+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:52:26.681+05:30 [Info] switched currmeta from 1629 -> 1629 force true 
2019-12-10T22:52:26.683+05:30 [Info] switched currmeta from 1633 -> 1633 force true 
2019-12-10T22:52:29.798+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] active conns 0, free conns 1
2019-12-10T22:52:29.869+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] active conns 0, free conns 1
2019-12-10T22:52:31.461+05:30 [Info] Rollback time has changed for index inst 11699873349970432747. New rollback time 1575998546674704467
2019-12-10T22:52:31.461+05:30 [Info] Rollback time has changed for index inst 11699873349970432747. New rollback time 1575998546674704467
2019-12-10T22:52:31.630+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:52:31.658+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:52:31.658+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:52:31.659+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:52:31.659+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:52:31.660+05:30 [Info] switched currmeta from 1633 -> 1634 force true 
2019-12-10T22:52:31.662+05:30 [Info] switched currmeta from 1629 -> 1630 force true 
2019-12-10T22:52:38.006+05:30 [Info] connected with 1 indexers
2019-12-10T22:52:38.006+05:30 [Info] index 6337225028508673115 has replicas [11699873349970432747]: 
2019-12-10T22:52:38.006+05:30 [Info] index 6337225028508673115 has equivalents [6337225028508673115]: 
2019-12-10T22:52:38.006+05:30 [Info] client load stats {"11699873349970432747": 2.172584e+06}
2019-12-10T22:52:38.006+05:30 [Info] client hit stats {"11699873349970432747": 1}
2019-12-10T22:52:38.006+05:30 [Info] client pending item stats {"11699873349970432747": 9223372036854775807}
2019-12-10T22:52:38.006+05:30 [Info] client stats current {"11699873349970432747": true}
2019-12-10T22:52:38.009+05:30 [Info] num concurrent scans {0}
2019-12-10T22:52:38.009+05:30 [Info] average scan response {3 ms}
2019-12-10T22:52:55.866+05:30 [Info] connected with 1 indexers
2019-12-10T22:52:55.866+05:30 [Info] index 6337225028508673115 has replicas [11699873349970432747]: 
2019-12-10T22:52:55.866+05:30 [Info] index 6337225028508673115 has equivalents [6337225028508673115]: 
2019-12-10T22:52:55.866+05:30 [Info] client load stats {"11699873349970432747": 0}
2019-12-10T22:52:55.866+05:30 [Info] client hit stats {"11699873349970432747": 0}
2019-12-10T22:52:55.866+05:30 [Info] client pending item stats {"11699873349970432747": 9223372036854775807}
2019-12-10T22:52:55.866+05:30 [Info] client stats current {"11699873349970432747": true}
2019-12-10T22:52:55.874+05:30 [Info] num concurrent scans {0}
2019-12-10T22:52:55.874+05:30 [Info] average scan response {4 ms}
--- PASS: TestIdxCorruptBackup (33.86s)
=== RUN   TestStatsPersistence
2019/12/10 22:52:56 In TestStatsPersistence()
2019/12/10 22:52:56 In DropAllSecondaryIndexes()
2019/12/10 22:52:56 Index found:  corrupt_idx2_company
2019-12-10T22:52:56.084+05:30 [Info] metadata provider version changed 1634 -> 1636
2019-12-10T22:52:56.084+05:30 [Info] switched currmeta from 1634 -> 1636 force false 
2019-12-10T22:52:56.084+05:30 [Info] DropIndex 6337225028508673115 - elapsed(81.222483ms), err()
2019/12/10 22:52:56 Dropped index corrupt_idx2_company
2019-12-10T22:52:56.218+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:52:56.218+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:52:56.221+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:52:56.222+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:52:56.222+05:30 [Info] switched currmeta from 1636 -> 1636 force true 
2019-12-10T22:52:56.230+05:30 [Info] switched currmeta from 1630 -> 1632 force true 
2019-12-10T22:52:56.297+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:52:56.297+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:52:56.306+05:30 [Info] switched currmeta from 1636 -> 1636 force true 
2019-12-10T22:52:56.318+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:52:56.318+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:52:56.336+05:30 [Info] switched currmeta from 1632 -> 1632 force true 
2019-12-10T22:52:56.953+05:30 [Info] send prepare create request to watcher 127.0.0.1:9106
2019-12-10T22:52:56.954+05:30 [Info] Indexer 127.0.0.1:9106 accept prepare request. Index (default, index_age)
2019-12-10T22:52:57.008+05:30 [Info] Planner::processCreateToken: There is no create token to process for node 127.0.0.1:9001
2019-12-10T22:52:57.069+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node 127.0.0.1:9001
2019-12-10T22:52:57.146+05:30 [Info] ************ Indexer Layout *************
2019-12-10T22:52:57.146+05:30 [Info] Score: 0
2019-12-10T22:52:57.146+05:30 [Info] variation: 0
2019-12-10T22:52:57.146+05:30 [Info] ElapsedTime: 5ms
2019-12-10T22:52:57.146+05:30 [Info] ConvergenceTime: 2us
2019-12-10T22:52:57.146+05:30 [Info] Iteration: 0
2019-12-10T22:52:57.146+05:30 [Info] Move: 0
2019-12-10T22:52:57.146+05:30 [Info] ----------------------------------------
2019-12-10T22:52:57.146+05:30 [Info] Indexer Memory Mean 1258291200 (1.17188G)
2019-12-10T22:52:57.146+05:30 [Info] Indexer Memory Deviation 0 (0) (0.00%)
2019-12-10T22:52:57.146+05:30 [Info] Indexer Memory Utilization 0.8000
2019-12-10T22:52:57.146+05:30 [Info] Indexer CPU Mean 0.0000
2019-12-10T22:52:57.146+05:30 [Info] Indexer CPU Deviation 0.00 (0.00%)
2019-12-10T22:52:57.146+05:30 [Info] Indexer CPU Utilization 0.0000
2019-12-10T22:52:57.146+05:30 [Info] Indexer IO Mean 0.0000
2019-12-10T22:52:57.146+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2019-12-10T22:52:57.146+05:30 [Info] Indexer Drain Rate Mean 0.0000
2019-12-10T22:52:57.146+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2019-12-10T22:52:57.146+05:30 [Info] Indexer Scan Rate Mean 0.0000
2019-12-10T22:52:57.146+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2019-12-10T22:52:57.146+05:30 [Info] Indexer Data Size Mean 1258291200 (1.17188G)
2019-12-10T22:52:57.146+05:30 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2019-12-10T22:52:57.146+05:30 [Info] Total Index Data (from non-deleted node) 0
2019-12-10T22:52:57.146+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2019-12-10T22:52:57.146+05:30 [Info] No. Index (from non-deleted node) 0
2019-12-10T22:52:57.146+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2019-12-10T22:52:57.146+05:30 [Info] ----------------------------------------
2019-12-10T22:52:57.146+05:30 [Info] Number of indexes: 1
2019-12-10T22:52:57.146+05:30 [Info] Number of indexers: 1
2019-12-10T22:52:57.146+05:30 [Info] Avg Index Size: 1258291200 (1.17188G)
2019-12-10T22:52:57.146+05:30 [Info] Max Index Size: 1258291200 (1.17188G)
2019-12-10T22:52:57.146+05:30 [Info] Max Indexer Overhead: 0 (0)
2019-12-10T22:52:57.146+05:30 [Info] Avg Index Cpu: 0.0000
2019-12-10T22:52:57.146+05:30 [Info] Max Index Cpu: 0.0000
2019-12-10T22:52:57.146+05:30 [Info] Num Estimation Run: 1
2019-12-10T22:52:57.146+05:30 [Info] ----------------------------------------
2019-12-10T22:52:57.146+05:30 [Info] Memory Quota 1572864000 (1.46484G)
2019-12-10T22:52:57.146+05:30 [Info] CPU Quota 4
2019-12-10T22:52:57.146+05:30 [Info] Max Cpu Utilization -1
2019-12-10T22:52:57.146+05:30 [Info] Max Memory Utilization -1
2019-12-10T22:52:57.146+05:30 [Info] ----------------------------------------
2019-12-10T22:52:57.146+05:30 [Info] Total iteration: 0
2019-12-10T22:52:57.146+05:30 [Info] RandomSwap time: 0ns
2019-12-10T22:52:57.146+05:30 [Info] RandomSwap call: 0
2019-12-10T22:52:57.146+05:30 [Info] RandomSwap iteration: 0
2019-12-10T22:52:57.146+05:30 [Info] RandomMove time: 0ns
2019-12-10T22:52:57.146+05:30 [Info] RandomMove call: 0 (empty index 0)
2019-12-10T22:52:57.146+05:30 [Info] ExhaustMove time: 0ns
2019-12-10T22:52:57.146+05:30 [Info] ExhaustMove call: 0
2019-12-10T22:52:57.146+05:30 [Info] ExhaustSwap time: 0ns
2019-12-10T22:52:57.146+05:30 [Info] ExhaustSwap call: 0
2019-12-10T22:52:57.146+05:30 [Info] ----------------------------------------
2019-12-10T22:52:57.146+05:30 [Info] 
2019-12-10T22:52:57.146+05:30 [Info] Indexer serverGroup:Group 1, nodeId:127.0.0.1:9001, nodeUUID:f7d69a0ebb5dfec6098045fb31c56ad5, useLiveData:true
2019-12-10T22:52:57.146+05:30 [Info] Indexer total memory:1258291200 (1.17188G), mem:1258291200 (1.17188G), overhead:0 (0), min:1258291200 (1.17188G), data:1258291200 (1.17188G) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:1
2019-12-10T22:52:57.146+05:30 [Info] Indexer isDeleted:false isNew:false exclude: meetConstraint:true
2019-12-10T22:52:57.146+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:52:57.146+05:30 [Info] 		Index name:index_age 0, bucket:default, defnId:17774760033278588180, instId:7460965192187187154, Partition: 0, new/moved:true
2019-12-10T22:52:57.146+05:30 [Info] 		Index total memory:1258291200 (1.17188G), mem:1258291200 (1.17188G), overhead:0 (0), min:1258291200 (1.17188G), data:1258291200 (1.17188G) cpu:0.0000 io:0 (0) scan:0 drain:0
2019-12-10T22:52:57.147+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:52:57.147+05:30 [Info] ****************************************
2019-12-10T22:52:57.147+05:30 [Info] send commit create request to watcher 127.0.0.1:9106
2019-12-10T22:52:57.305+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:52:57.305+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:52:57.312+05:30 [Info] switched currmeta from 1636 -> 1638 force true 
2019-12-10T22:52:57.315+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:52:57.315+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:52:57.319+05:30 [Info] switched currmeta from 1632 -> 1634 force true 
2019-12-10T22:52:57.412+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:52:57.412+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:52:57.415+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:52:57.415+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:52:57.416+05:30 [Info] switched currmeta from 1634 -> 1636 force true 
2019-12-10T22:52:57.421+05:30 [Info] switched currmeta from 1638 -> 1640 force true 
2019-12-10T22:52:57.463+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:52:57.463+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:52:57.464+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:52:57.464+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:52:57.472+05:30 [Info] switched currmeta from 1640 -> 1640 force true 
2019-12-10T22:52:57.475+05:30 [Info] switched currmeta from 1636 -> 1636 force true 
2019-12-10T22:52:57.627+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:52:57.627+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:52:57.632+05:30 [Info] switched currmeta from 1636 -> 1637 force true 
2019-12-10T22:52:57.633+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:52:57.633+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:52:57.639+05:30 [Info] switched currmeta from 1640 -> 1641 force true 
2019-12-10T22:52:59.959+05:30 [Info] CreateIndex 17774760033278588180 default/index_age using:forestdb exprType:N1QL whereExpr:() secExprs:([`age`]) desc:[] isPrimary:false scheme:SINGLE  partitionKeys:([]) with: - elapsed(3.006182074s) err()
2019/12/10 22:52:59 Created the secondary index index_age. Waiting for it become active
2019-12-10T22:52:59.960+05:30 [Info] metadata provider version changed 1641 -> 1642
2019-12-10T22:52:59.960+05:30 [Info] switched currmeta from 1641 -> 1642 force false 
2019/12/10 22:52:59 Index is now active
2019-12-10T22:52:59.960+05:30 [Info] send prepare create request to watcher 127.0.0.1:9106
2019-12-10T22:52:59.961+05:30 [Info] Indexer 127.0.0.1:9106 accept prepare request. Index (default, index_gender)
2019-12-10T22:53:00.006+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:53:00.006+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:53:00.011+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:53:00.011+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:53:00.013+05:30 [Info] switched currmeta from 1637 -> 1638 force true 
2019-12-10T22:53:00.015+05:30 [Info] switched currmeta from 1642 -> 1642 force true 
2019-12-10T22:53:00.041+05:30 [Info] Planner::processCreateToken: There is no create token to process for node 127.0.0.1:9001
2019-12-10T22:53:00.093+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node 127.0.0.1:9001
2019-12-10T22:53:00.172+05:30 [Info] ************ Indexer Layout *************
2019-12-10T22:53:00.172+05:30 [Info] Score: 0
2019-12-10T22:53:00.172+05:30 [Info] variation: 0
2019-12-10T22:53:00.172+05:30 [Info] ElapsedTime: 8ms
2019-12-10T22:53:00.172+05:30 [Info] ConvergenceTime: 2us
2019-12-10T22:53:00.172+05:30 [Info] Iteration: 0
2019-12-10T22:53:00.172+05:30 [Info] Move: 0
2019-12-10T22:53:00.172+05:30 [Info] ----------------------------------------
2019-12-10T22:53:00.172+05:30 [Info] Indexer Memory Mean 1258291200 (1.17188G)
2019-12-10T22:53:00.172+05:30 [Info] Indexer Memory Deviation 0 (0) (0.00%)
2019-12-10T22:53:00.172+05:30 [Info] Indexer Memory Utilization 0.8000
2019-12-10T22:53:00.172+05:30 [Info] Indexer CPU Mean 0.0000
2019-12-10T22:53:00.172+05:30 [Info] Indexer CPU Deviation 0.00 (0.00%)
2019-12-10T22:53:00.172+05:30 [Info] Indexer CPU Utilization 0.0000
2019-12-10T22:53:00.172+05:30 [Info] Indexer IO Mean 0.0000
2019-12-10T22:53:00.172+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2019-12-10T22:53:00.172+05:30 [Info] Indexer Drain Rate Mean 0.0000
2019-12-10T22:53:00.172+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2019-12-10T22:53:00.172+05:30 [Info] Indexer Scan Rate Mean 0.0000
2019-12-10T22:53:00.172+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2019-12-10T22:53:00.172+05:30 [Info] Indexer Data Size Mean 1258645598 (1.17221G)
2019-12-10T22:53:00.172+05:30 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2019-12-10T22:53:00.172+05:30 [Info] Total Index Data (from non-deleted node) 0
2019-12-10T22:53:00.172+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2019-12-10T22:53:00.172+05:30 [Info] No. Index (from non-deleted node) 0
2019-12-10T22:53:00.172+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2019-12-10T22:53:00.172+05:30 [Info] ----------------------------------------
2019-12-10T22:53:00.172+05:30 [Info] Number of indexes: 2
2019-12-10T22:53:00.172+05:30 [Info] Number of indexers: 1
2019-12-10T22:53:00.172+05:30 [Info] Avg Index Size: 629145600 (600M)
2019-12-10T22:53:00.172+05:30 [Info] Max Index Size: 1258291200 (1.17188G)
2019-12-10T22:53:00.172+05:30 [Info] Max Indexer Overhead: 0 (0)
2019-12-10T22:53:00.172+05:30 [Info] Avg Index Cpu: 0.0000
2019-12-10T22:53:00.172+05:30 [Info] Max Index Cpu: 0.0000
2019-12-10T22:53:00.172+05:30 [Info] Num Estimation Run: 1
2019-12-10T22:53:00.172+05:30 [Info] ----------------------------------------
2019-12-10T22:53:00.172+05:30 [Info] Memory Quota 1572864000 (1.46484G)
2019-12-10T22:53:00.172+05:30 [Info] CPU Quota 4
2019-12-10T22:53:00.172+05:30 [Info] Max Cpu Utilization -1
2019-12-10T22:53:00.172+05:30 [Info] Max Memory Utilization -1
2019-12-10T22:53:00.172+05:30 [Info] ----------------------------------------
2019-12-10T22:53:00.172+05:30 [Info] Total iteration: 0
2019-12-10T22:53:00.172+05:30 [Info] RandomSwap time: 0ns
2019-12-10T22:53:00.172+05:30 [Info] RandomSwap call: 0
2019-12-10T22:53:00.172+05:30 [Info] RandomSwap iteration: 0
2019-12-10T22:53:00.172+05:30 [Info] RandomMove time: 0ns
2019-12-10T22:53:00.172+05:30 [Info] RandomMove call: 0 (empty index 0)
2019-12-10T22:53:00.172+05:30 [Info] ExhaustMove time: 0ns
2019-12-10T22:53:00.172+05:30 [Info] ExhaustMove call: 0
2019-12-10T22:53:00.172+05:30 [Info] ExhaustSwap time: 0ns
2019-12-10T22:53:00.172+05:30 [Info] ExhaustSwap call: 0
2019-12-10T22:53:00.172+05:30 [Info] ----------------------------------------
2019-12-10T22:53:00.172+05:30 [Info] 
2019-12-10T22:53:00.172+05:30 [Info] Indexer serverGroup:Group 1, nodeId:127.0.0.1:9001, nodeUUID:f7d69a0ebb5dfec6098045fb31c56ad5, useLiveData:true
2019-12-10T22:53:00.172+05:30 [Info] Indexer total memory:1258291200 (1.17188G), mem:1258291200 (1.17188G), overhead:0 (0), min:1258362079 (1.17194G), data:1258645598 (1.17221G) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:2
2019-12-10T22:53:00.173+05:30 [Info] Indexer isDeleted:false isNew:false exclude: meetConstraint:true
2019-12-10T22:53:00.173+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:53:00.173+05:30 [Info] 		Index name:index_age 0, bucket:default, defnId:17774760033278588180, instId:12601004226662024070, Partition: 0, new/moved:false
2019-12-10T22:53:00.173+05:30 [Info] 		Index total memory:0 (0), mem:0 (0), overhead:0 (0), min:70879 (69.2178K), data:354398 (346.092K) cpu:0.0000 io:0 (0) scan:0 drain:0
2019-12-10T22:53:00.173+05:30 [Info] 		Index resident:0% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:53:00.173+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:53:00.173+05:30 [Info] 		Index name:index_gender 0, bucket:default, defnId:8931850466729319815, instId:11920200779802055459, Partition: 0, new/moved:true
2019-12-10T22:53:00.173+05:30 [Info] 		Index total memory:1258291200 (1.17188G), mem:1258291200 (1.17188G), overhead:0 (0), min:1258291200 (1.17188G), data:1258291200 (1.17188G) cpu:0.0000 io:0 (0) scan:0 drain:0
2019-12-10T22:53:00.173+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:53:00.173+05:30 [Info] ****************************************
2019-12-10T22:53:00.173+05:30 [Info] send commit create request to watcher 127.0.0.1:9106
2019-12-10T22:53:00.250+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:53:00.250+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:53:00.254+05:30 [Info] switched currmeta from 1638 -> 1640 force true 
2019-12-10T22:53:00.257+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:53:00.257+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:53:00.260+05:30 [Info] switched currmeta from 1642 -> 1644 force true 
2019-12-10T22:53:00.358+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:53:00.358+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:53:00.364+05:30 [Info] switched currmeta from 1640 -> 1642 force true 
2019-12-10T22:53:00.372+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:53:00.372+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:53:00.374+05:30 [Info] switched currmeta from 1644 -> 1646 force true 
2019-12-10T22:53:00.412+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:53:00.412+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:53:00.418+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:53:00.418+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:53:00.419+05:30 [Info] switched currmeta from 1642 -> 1642 force true 
2019-12-10T22:53:00.420+05:30 [Info] switched currmeta from 1646 -> 1646 force true 
2019-12-10T22:53:00.534+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:53:00.534+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:53:00.539+05:30 [Info] switched currmeta from 1642 -> 1643 force true 
2019-12-10T22:53:00.541+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:53:00.541+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:53:00.545+05:30 [Info] switched currmeta from 1646 -> 1647 force true 
2019-12-10T22:53:01.464+05:30 [Info] Rollback time has changed for index inst 12601004226662024070. New rollback time 1575998546674704467
2019-12-10T22:53:01.465+05:30 [Info] Rollback time has changed for index inst 12601004226662024070. New rollback time 1575998546674704467
2019-12-10T22:53:01.657+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:53:01.748+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:53:01.748+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:53:01.752+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:53:01.752+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:53:01.758+05:30 [Info] switched currmeta from 1643 -> 1643 force true 
2019-12-10T22:53:01.759+05:30 [Info] switched currmeta from 1647 -> 1647 force true 
2019-12-10T22:53:02.948+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:53:02.948+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:53:02.953+05:30 [Info] switched currmeta from 1643 -> 1644 force true 
2019-12-10T22:53:02.953+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:53:02.953+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:53:02.959+05:30 [Info] switched currmeta from 1647 -> 1648 force true 
2019-12-10T22:53:03.669+05:30 [Info] CreateIndex 8931850466729319815 default/index_gender using:forestdb exprType:N1QL whereExpr:() secExprs:([`gender`]) desc:[] isPrimary:false scheme:SINGLE  partitionKeys:([]) with: - elapsed(3.708640007s) err()
2019/12/10 22:53:03 Created the secondary index index_gender. Waiting for it become active
2019-12-10T22:53:03.669+05:30 [Info] metadata provider version changed 1648 -> 1649
2019-12-10T22:53:03.669+05:30 [Info] switched currmeta from 1648 -> 1649 force false 
2019/12/10 22:53:03 Index is now active
2019-12-10T22:53:03.669+05:30 [Info] send prepare create request to watcher 127.0.0.1:9106
2019-12-10T22:53:03.670+05:30 [Info] Indexer 127.0.0.1:9106 accept prepare request. Index (default, index_city)
2019-12-10T22:53:03.756+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:53:03.756+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:53:03.756+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:53:03.756+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:53:03.763+05:30 [Info] switched currmeta from 1644 -> 1645 force true 
2019-12-10T22:53:03.763+05:30 [Info] switched currmeta from 1649 -> 1649 force true 
2019-12-10T22:53:03.825+05:30 [Info] Planner::processCreateToken: There is no create token to process for node 127.0.0.1:9001
2019-12-10T22:53:03.891+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node 127.0.0.1:9001
2019-12-10T22:53:03.982+05:30 [Info] ************ Indexer Layout *************
2019-12-10T22:53:03.982+05:30 [Info] Score: 0
2019-12-10T22:53:03.982+05:30 [Info] variation: 0
2019-12-10T22:53:03.982+05:30 [Info] ElapsedTime: 8ms
2019-12-10T22:53:03.982+05:30 [Info] ConvergenceTime: 2us
2019-12-10T22:53:03.982+05:30 [Info] Iteration: 0
2019-12-10T22:53:03.982+05:30 [Info] Move: 0
2019-12-10T22:53:03.982+05:30 [Info] ----------------------------------------
2019-12-10T22:53:03.982+05:30 [Info] Indexer Memory Mean 1258291200 (1.17188G)
2019-12-10T22:53:03.982+05:30 [Info] Indexer Memory Deviation 0 (0) (0.00%)
2019-12-10T22:53:03.982+05:30 [Info] Indexer Memory Utilization 0.8000
2019-12-10T22:53:03.982+05:30 [Info] Indexer CPU Mean 0.0000
2019-12-10T22:53:03.982+05:30 [Info] Indexer CPU Deviation 0.00 (0.00%)
2019-12-10T22:53:03.982+05:30 [Info] Indexer CPU Utilization 0.0000
2019-12-10T22:53:03.982+05:30 [Info] Indexer IO Mean 0.0000
2019-12-10T22:53:03.982+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2019-12-10T22:53:03.982+05:30 [Info] Indexer Drain Rate Mean 0.0000
2019-12-10T22:53:03.982+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2019-12-10T22:53:03.982+05:30 [Info] Indexer Scan Rate Mean 0.0000
2019-12-10T22:53:03.982+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2019-12-10T22:53:03.982+05:30 [Info] Indexer Data Size Mean 1258979826 (1.17252G)
2019-12-10T22:53:03.982+05:30 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2019-12-10T22:53:03.982+05:30 [Info] Total Index Data (from non-deleted node) 0
2019-12-10T22:53:03.982+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2019-12-10T22:53:03.982+05:30 [Info] No. Index (from non-deleted node) 0
2019-12-10T22:53:03.982+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2019-12-10T22:53:03.982+05:30 [Info] ----------------------------------------
2019-12-10T22:53:03.982+05:30 [Info] Number of indexes: 3
2019-12-10T22:53:03.982+05:30 [Info] Number of indexers: 1
2019-12-10T22:53:03.982+05:30 [Info] Avg Index Size: 419430400 (400M)
2019-12-10T22:53:03.982+05:30 [Info] Max Index Size: 1258291200 (1.17188G)
2019-12-10T22:53:03.982+05:30 [Info] Max Indexer Overhead: 0 (0)
2019-12-10T22:53:03.982+05:30 [Info] Avg Index Cpu: 0.0000
2019-12-10T22:53:03.982+05:30 [Info] Max Index Cpu: 0.0000
2019-12-10T22:53:03.982+05:30 [Info] Num Estimation Run: 1
2019-12-10T22:53:03.982+05:30 [Info] ----------------------------------------
2019-12-10T22:53:03.982+05:30 [Info] Memory Quota 1572864000 (1.46484G)
2019-12-10T22:53:03.982+05:30 [Info] CPU Quota 4
2019-12-10T22:53:03.982+05:30 [Info] Max Cpu Utilization -1
2019-12-10T22:53:03.982+05:30 [Info] Max Memory Utilization -1
2019-12-10T22:53:03.982+05:30 [Info] ----------------------------------------
2019-12-10T22:53:03.982+05:30 [Info] Total iteration: 0
2019-12-10T22:53:03.982+05:30 [Info] RandomSwap time: 0ns
2019-12-10T22:53:03.983+05:30 [Info] RandomSwap call: 0
2019-12-10T22:53:03.983+05:30 [Info] RandomSwap iteration: 0
2019-12-10T22:53:03.983+05:30 [Info] RandomMove time: 0ns
2019-12-10T22:53:03.983+05:30 [Info] RandomMove call: 0 (empty index 0)
2019-12-10T22:53:03.983+05:30 [Info] ExhaustMove time: 0ns
2019-12-10T22:53:03.983+05:30 [Info] ExhaustMove call: 0
2019-12-10T22:53:03.983+05:30 [Info] ExhaustSwap time: 0ns
2019-12-10T22:53:03.983+05:30 [Info] ExhaustSwap call: 0
2019-12-10T22:53:03.983+05:30 [Info] ----------------------------------------
2019-12-10T22:53:03.983+05:30 [Info] 
2019-12-10T22:53:03.983+05:30 [Info] Indexer serverGroup:Group 1, nodeId:127.0.0.1:9001, nodeUUID:f7d69a0ebb5dfec6098045fb31c56ad5, useLiveData:true
2019-12-10T22:53:03.983+05:30 [Info] Indexer total memory:1258291200 (1.17188G), mem:1258291200 (1.17188G), overhead:0 (0), min:1258428924 (1.172G), data:1258979826 (1.17252G) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:3
2019-12-10T22:53:03.983+05:30 [Info] Indexer isDeleted:false isNew:false exclude: meetConstraint:true
2019-12-10T22:53:03.983+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:53:03.983+05:30 [Info] 		Index name:index_age 0, bucket:default, defnId:17774760033278588180, instId:12601004226662024070, Partition: 0, new/moved:false
2019-12-10T22:53:03.983+05:30 [Info] 		Index total memory:0 (0), mem:0 (0), overhead:0 (0), min:70879 (69.2178K), data:354398 (346.092K) cpu:0.0000 io:0 (0) scan:0 drain:0
2019-12-10T22:53:03.983+05:30 [Info] 		Index resident:0% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:53:03.983+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:53:03.983+05:30 [Info] 		Index name:index_gender 0, bucket:default, defnId:8931850466729319815, instId:2136088608776327684, Partition: 0, new/moved:false
2019-12-10T22:53:03.983+05:30 [Info] 		Index total memory:0 (0), mem:0 (0), overhead:0 (0), min:66845 (65.2783K), data:334228 (326.395K) cpu:0.0000 io:0 (0) scan:0 drain:0
2019-12-10T22:53:03.983+05:30 [Info] 		Index resident:0% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:53:03.983+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:53:03.983+05:30 [Info] 		Index name:index_city 0, bucket:default, defnId:11728303485226421575, instId:15019110612963707204, Partition: 0, new/moved:true
2019-12-10T22:53:03.983+05:30 [Info] 		Index total memory:1258291200 (1.17188G), mem:1258291200 (1.17188G), overhead:0 (0), min:1258291200 (1.17188G), data:1258291200 (1.17188G) cpu:0.0000 io:0 (0) scan:0 drain:0
2019-12-10T22:53:03.983+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:53:03.983+05:30 [Info] ****************************************
2019-12-10T22:53:03.983+05:30 [Info] send commit create request to watcher 127.0.0.1:9106
2019-12-10T22:53:04.056+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:53:04.056+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:53:04.065+05:30 [Info] switched currmeta from 1645 -> 1647 force true 
2019-12-10T22:53:04.075+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:53:04.075+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:53:04.077+05:30 [Info] switched currmeta from 1649 -> 1651 force true 
2019-12-10T22:53:04.158+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:53:04.158+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:53:04.162+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:53:04.162+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:53:04.167+05:30 [Info] switched currmeta from 1651 -> 1653 force true 
2019-12-10T22:53:04.169+05:30 [Info] switched currmeta from 1647 -> 1649 force true 
2019-12-10T22:53:04.213+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:53:04.214+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:53:04.219+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:53:04.219+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:53:04.223+05:30 [Info] switched currmeta from 1649 -> 1649 force true 
2019-12-10T22:53:04.226+05:30 [Info] switched currmeta from 1653 -> 1653 force true 
2019-12-10T22:53:04.337+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:53:04.337+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:53:04.341+05:30 [Info] switched currmeta from 1649 -> 1650 force true 
2019-12-10T22:53:04.342+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:53:04.342+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:53:04.348+05:30 [Info] switched currmeta from 1653 -> 1654 force true 
2019-12-10T22:53:06.461+05:30 [Info] Rollback time has changed for index inst 2136088608776327684. New rollback time 1575998546674704467
2019-12-10T22:53:06.461+05:30 [Info] Rollback time has changed for index inst 2136088608776327684. New rollback time 1575998546674704467
2019-12-10T22:53:06.657+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:53:06.695+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:53:06.695+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:53:06.695+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:53:06.695+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:53:06.697+05:30 [Info] switched currmeta from 1654 -> 1654 force true 
2019-12-10T22:53:06.697+05:30 [Info] switched currmeta from 1650 -> 1650 force true 
2019-12-10T22:53:06.957+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:53:06.957+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:53:06.957+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:53:06.957+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:53:06.960+05:30 [Info] switched currmeta from 1650 -> 1651 force true 
2019-12-10T22:53:06.962+05:30 [Info] switched currmeta from 1654 -> 1655 force true 
2019-12-10T22:53:07.425+05:30 [Info] CreateIndex 11728303485226421575 default/index_city using:forestdb exprType:N1QL whereExpr:() secExprs:([(`address`.`city`)]) desc:[] isPrimary:false scheme:SINGLE  partitionKeys:([]) with: - elapsed(3.75553668s) err()
2019/12/10 22:53:07 Created the secondary index index_city. Waiting for it become active
2019-12-10T22:53:07.425+05:30 [Info] metadata provider version changed 1655 -> 1656
2019-12-10T22:53:07.425+05:30 [Info] switched currmeta from 1655 -> 1656 force false 
2019/12/10 22:53:07 Index is now active
2019-12-10T22:53:07.425+05:30 [Info] send prepare create request to watcher 127.0.0.1:9106
2019-12-10T22:53:07.426+05:30 [Info] Indexer 127.0.0.1:9106 accept prepare request. Index (default, p1)
2019-12-10T22:53:07.511+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:53:07.511+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:53:07.512+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:53:07.512+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:53:07.525+05:30 [Info] switched currmeta from 1656 -> 1656 force true 
2019-12-10T22:53:07.526+05:30 [Info] switched currmeta from 1651 -> 1652 force true 
2019-12-10T22:53:07.575+05:30 [Info] Planner::processCreateToken: There is no create token to process for node 127.0.0.1:9001
2019-12-10T22:53:07.622+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node 127.0.0.1:9001
2019-12-10T22:53:07.807+05:30 [Info] ************ Indexer Layout *************
2019-12-10T22:53:07.807+05:30 [Info] Score: 0
2019-12-10T22:53:07.807+05:30 [Info] variation: 0
2019-12-10T22:53:07.807+05:30 [Info] ElapsedTime: 90ms
2019-12-10T22:53:07.807+05:30 [Info] ConvergenceTime: 2us
2019-12-10T22:53:07.807+05:30 [Info] Iteration: 0
2019-12-10T22:53:07.807+05:30 [Info] Move: 0
2019-12-10T22:53:07.807+05:30 [Info] ----------------------------------------
2019-12-10T22:53:07.807+05:30 [Info] Indexer Memory Mean 1258291200 (1.17188G)
2019-12-10T22:53:07.807+05:30 [Info] Indexer Memory Deviation 0 (0) (0.00%)
2019-12-10T22:53:07.807+05:30 [Info] Indexer Memory Utilization 0.8000
2019-12-10T22:53:07.807+05:30 [Info] Indexer CPU Mean 0.0000
2019-12-10T22:53:07.807+05:30 [Info] Indexer CPU Deviation 0.00 (0.00%)
2019-12-10T22:53:07.807+05:30 [Info] Indexer CPU Utilization 0.0000
2019-12-10T22:53:07.807+05:30 [Info] Indexer IO Mean 0.0000
2019-12-10T22:53:07.807+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2019-12-10T22:53:07.807+05:30 [Info] Indexer Drain Rate Mean 0.0000
2019-12-10T22:53:07.807+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2019-12-10T22:53:07.807+05:30 [Info] Indexer Scan Rate Mean 0.0000
2019-12-10T22:53:07.807+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2019-12-10T22:53:07.807+05:30 [Info] Indexer Data Size Mean 1259320482 (1.17283G)
2019-12-10T22:53:07.807+05:30 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2019-12-10T22:53:07.807+05:30 [Info] Total Index Data (from non-deleted node) 0
2019-12-10T22:53:07.807+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2019-12-10T22:53:07.807+05:30 [Info] No. Index (from non-deleted node) 0
2019-12-10T22:53:07.807+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2019-12-10T22:53:07.807+05:30 [Info] ----------------------------------------
2019-12-10T22:53:07.807+05:30 [Info] Number of indexes: 4
2019-12-10T22:53:07.807+05:30 [Info] Number of indexers: 1
2019-12-10T22:53:07.807+05:30 [Info] Avg Index Size: 314572800 (300M)
2019-12-10T22:53:07.807+05:30 [Info] Max Index Size: 1258291200 (1.17188G)
2019-12-10T22:53:07.807+05:30 [Info] Max Indexer Overhead: 0 (0)
2019-12-10T22:53:07.807+05:30 [Info] Avg Index Cpu: 0.0000
2019-12-10T22:53:07.807+05:30 [Info] Max Index Cpu: 0.0000
2019-12-10T22:53:07.807+05:30 [Info] Num Estimation Run: 1
2019-12-10T22:53:07.807+05:30 [Info] ----------------------------------------
2019-12-10T22:53:07.807+05:30 [Info] Memory Quota 1572864000 (1.46484G)
2019-12-10T22:53:07.807+05:30 [Info] CPU Quota 4
2019-12-10T22:53:07.807+05:30 [Info] Max Cpu Utilization -1
2019-12-10T22:53:07.807+05:30 [Info] Max Memory Utilization -1
2019-12-10T22:53:07.807+05:30 [Info] ----------------------------------------
2019-12-10T22:53:07.807+05:30 [Info] Total iteration: 0
2019-12-10T22:53:07.807+05:30 [Info] RandomSwap time: 0ns
2019-12-10T22:53:07.807+05:30 [Info] RandomSwap call: 0
2019-12-10T22:53:07.807+05:30 [Info] RandomSwap iteration: 0
2019-12-10T22:53:07.807+05:30 [Info] RandomMove time: 0ns
2019-12-10T22:53:07.807+05:30 [Info] RandomMove call: 0 (empty index 0)
2019-12-10T22:53:07.808+05:30 [Info] ExhaustMove time: 0ns
2019-12-10T22:53:07.808+05:30 [Info] ExhaustMove call: 0
2019-12-10T22:53:07.808+05:30 [Info] ExhaustSwap time: 0ns
2019-12-10T22:53:07.808+05:30 [Info] ExhaustSwap call: 0
2019-12-10T22:53:07.808+05:30 [Info] ----------------------------------------
2019-12-10T22:53:07.808+05:30 [Info] 
2019-12-10T22:53:07.808+05:30 [Info] Indexer serverGroup:Group 1, nodeId:127.0.0.1:9001, nodeUUID:f7d69a0ebb5dfec6098045fb31c56ad5, useLiveData:true
2019-12-10T22:53:07.808+05:30 [Info] Indexer total memory:1258291200 (1.17188G), mem:1258291200 (1.17188G), overhead:0 (0), min:1258497055 (1.17207G), data:1259320482 (1.17283G) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:4
2019-12-10T22:53:07.808+05:30 [Info] Indexer isDeleted:false isNew:false exclude: meetConstraint:true
2019-12-10T22:53:07.808+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:53:07.808+05:30 [Info] 		Index name:index_age 0, bucket:default, defnId:17774760033278588180, instId:12601004226662024070, Partition: 0, new/moved:false
2019-12-10T22:53:07.808+05:30 [Info] 		Index total memory:0 (0), mem:0 (0), overhead:0 (0), min:70879 (69.2178K), data:354398 (346.092K) cpu:0.0000 io:0 (0) scan:0 drain:0
2019-12-10T22:53:07.808+05:30 [Info] 		Index resident:0% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:53:07.808+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:53:07.808+05:30 [Info] 		Index name:index_gender 0, bucket:default, defnId:8931850466729319815, instId:2136088608776327684, Partition: 0, new/moved:false
2019-12-10T22:53:07.808+05:30 [Info] 		Index total memory:0 (0), mem:0 (0), overhead:0 (0), min:66845 (65.2783K), data:334228 (326.395K) cpu:0.0000 io:0 (0) scan:0 drain:0
2019-12-10T22:53:07.808+05:30 [Info] 		Index resident:0% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:53:07.808+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:53:07.808+05:30 [Info] 		Index name:index_city 0, bucket:default, defnId:11728303485226421575, instId:2576283963928241955, Partition: 0, new/moved:false
2019-12-10T22:53:07.808+05:30 [Info] 		Index total memory:0 (0), mem:0 (0), overhead:0 (0), min:68131 (66.5342K), data:340656 (332.672K) cpu:0.0000 io:0 (0) scan:0 drain:0
2019-12-10T22:53:07.808+05:30 [Info] 		Index resident:0% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:53:07.808+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2019-12-10T22:53:07.808+05:30 [Info] 		Index name:p1 0, bucket:default, defnId:8467595592936537298, instId:11030095845704595825, Partition: 0, new/moved:true
2019-12-10T22:53:07.808+05:30 [Info] 		Index total memory:1258291200 (1.17188G), mem:1258291200 (1.17188G), overhead:0 (0), min:1258291200 (1.17188G), data:1258291200 (1.17188G) cpu:0.0000 io:0 (0) scan:0 drain:0
2019-12-10T22:53:07.808+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2019-12-10T22:53:07.808+05:30 [Info] ****************************************
2019-12-10T22:53:07.808+05:30 [Info] send commit create request to watcher 127.0.0.1:9106
2019-12-10T22:53:07.893+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:53:07.894+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:53:07.899+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:53:07.899+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:53:07.903+05:30 [Info] switched currmeta from 1656 -> 1658 force true 
2019-12-10T22:53:07.904+05:30 [Info] switched currmeta from 1652 -> 1654 force true 
2019-12-10T22:53:08.009+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:53:08.009+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:53:08.009+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:53:08.009+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:53:08.012+05:30 [Info] switched currmeta from 1654 -> 1656 force true 
2019-12-10T22:53:08.014+05:30 [Info] switched currmeta from 1658 -> 1660 force true 
2019-12-10T22:53:08.064+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:53:08.065+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:53:08.066+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:53:08.066+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:53:08.071+05:30 [Info] switched currmeta from 1656 -> 1656 force true 
2019-12-10T22:53:08.073+05:30 [Info] switched currmeta from 1660 -> 1660 force true 
2019-12-10T22:53:08.178+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:53:08.178+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:53:08.180+05:30 [Info] switched currmeta from 1656 -> 1657 force true 
2019-12-10T22:53:08.182+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:53:08.182+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:53:08.186+05:30 [Info] switched currmeta from 1660 -> 1661 force true 
2019-12-10T22:53:10.417+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:53:10.417+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:53:10.421+05:30 [Info] switched currmeta from 1661 -> 1662 force true 
2019-12-10T22:53:10.429+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:53:10.429+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:53:10.431+05:30 [Info] switched currmeta from 1657 -> 1658 force true 
2019-12-10T22:53:11.231+05:30 [Info] CreateIndex 8467595592936537298 default/p1 using:forestdb exprType:N1QL whereExpr:() secExprs:([]) desc:[] isPrimary:true scheme:SINGLE  partitionKeys:([]) with: - elapsed(3.806667475s) err()
2019/12/10 22:53:11 Created the secondary index p1. Waiting for it become active
2019-12-10T22:53:11.232+05:30 [Info] metadata provider version changed 1662 -> 1663
2019-12-10T22:53:11.232+05:30 [Info] switched currmeta from 1662 -> 1663 force false 
2019/12/10 22:53:11 Index is now active
2019/12/10 22:53:11 === Testing for persistence enabled = true, with interval = 5  ===
2019-12-10T22:53:11.232+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9106 connected=true
2019/12/10 22:53:11 DBG: ChangeIndexerSettings: nodes = [0xc050c56800]
2019/12/10 22:53:11 DBG: ChangeIndexerSettings: adminurl = 127.0.0.1:9106 host 127.0.0.1 sport 9106
2019/12/10 22:53:11 Changing config key indexer.statsPersistenceInterval to value 5
2019/12/10 22:53:11 []
2019-12-10T22:53:11.331+05:30 [Info] New settings received: 
{"indexer.api.enableTestServer":true,"indexer.settings.allow_large_keys":false,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfFname":"","indexer.settings.cpuProfile":false,"indexer.settings.enable_corrupt_index_backup":true,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfFname":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":250,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.recovery.max_rollbacks":5,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":0,"indexer.settings.send_buffer_size":1024,"indexer.settings.sliceBufSize":50000,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"forestdb","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.wal_size":4096,"indexer.statsPersistenceInterval":5,"projector.settings.log_level":"info","queryport.client.settings.backfillLimit":0,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100}
2019-12-10T22:53:11.340+05:30 [Info] GSIC[default/default-1575992531337069731] logstats "default" {"gsi_scan_count":471,"gsi_scan_duration":84110305702,"gsi_throttle_duration":858278478,"gsi_prime_duration":6855890739,"gsi_blocked_duration":469005915,"gsi_total_temp_files":0}
2019-12-10T22:53:11.344+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:53:11.344+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:53:11.368+05:30 [Info] switched currmeta from 1663 -> 1663 force true 
2019-12-10T22:53:11.369+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:53:11.369+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:53:11.371+05:30 [Info] switched currmeta from 1658 -> 1659 force true 
2019-12-10T22:53:11.389+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:9106.  Error = EOF. Kill Pipe.
2019-12-10T22:53:11.389+05:30 [Error] WatcherServer.runOnce() : Watcher terminated unexpectedly.
2019-12-10T22:53:11.389+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:9106.  Error = EOF. Kill Pipe.
2019-12-10T22:53:11.390+05:30 [Error] WatcherServer.runOnce() : Watcher terminated unexpectedly.
2019-12-10T22:53:11.647+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:53:11.684+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:53:11.684+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:53:11.686+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:53:11.686+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:53:11.690+05:30 [Info] switched currmeta from 1663 -> 1663 force true 
2019-12-10T22:53:11.690+05:30 [Info] switched currmeta from 1659 -> 1659 force true 
2019/12/10 22:53:16 Using n1ql client
2019-12-10T22:53:16.331+05:30 [Info] GsiClient::UpdateUsecjson: using collatejson as data format between indexer and GsiClient
2019-12-10T22:53:16.332+05:30 [Error] transport error between 127.0.0.1:45408->127.0.0.1:9107: write tcp 127.0.0.1:45408->127.0.0.1:9107: write: broken pipe
2019-12-10T22:53:16.332+05:30 [Error] [GsiScanClient:"127.0.0.1:9107"] -5342126071128376126 request transport failed `write tcp 127.0.0.1:45408->127.0.0.1:9107: write: broken pipe`
2019-12-10T22:53:16.332+05:30 [Error] PickRandom: Fail to find indexer for all index partitions. Num partition 1.  Partition with instances 0 
2019/12/10 22:53:16 Using n1ql client
2019/12/10 22:53:16 Using n1ql client
2019/12/10 22:53:23 []
2019-12-10T22:53:23.476+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:9106.  Error = EOF. Kill Pipe.
2019-12-10T22:53:23.476+05:30 [Error] WatcherServer.runOnce() : Watcher terminated unexpectedly.
2019-12-10T22:53:23.476+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:9106.  Error = EOF. Kill Pipe.
2019-12-10T22:53:23.476+05:30 [Error] WatcherServer.runOnce() : Watcher terminated unexpectedly.
2019/12/10 22:53:28 === Testing for persistence enabled = false, with interval = 0  ===
2019/12/10 22:53:28 DBG: ChangeIndexerSettings: nodes = [0xc03d452880]
2019/12/10 22:53:28 DBG: ChangeIndexerSettings: adminurl = 127.0.0.1:9106 host 127.0.0.1 sport 9106
2019/12/10 22:53:28 Changing config key indexer.statsPersistenceInterval to value 0
2019/12/10 22:53:28 []
2019-12-10T22:53:29.815+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] active conns 0, free conns 1
2019-12-10T22:53:29.885+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] active conns 0, free conns 1
2019-12-10T22:53:31.667+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:53:31.698+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:53:31.698+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:53:31.699+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:53:31.699+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:53:31.701+05:30 [Info] switched currmeta from 1660 -> 1660 force true 
2019-12-10T22:53:31.703+05:30 [Info] switched currmeta from 1664 -> 1664 force true 
2019/12/10 22:53:33 Using n1ql client
2019-12-10T22:53:33.642+05:30 [Info] GsiClient::UpdateUsecjson: using collatejson as data format between indexer and GsiClient
2019-12-10T22:53:33.643+05:30 [Error] transport error between 127.0.0.1:46554->127.0.0.1:9107: write tcp 127.0.0.1:46554->127.0.0.1:9107: write: broken pipe
2019-12-10T22:53:33.643+05:30 [Error] [GsiScanClient:"127.0.0.1:9107"] -8299391430679456350 request transport failed `write tcp 127.0.0.1:46554->127.0.0.1:9107: write: broken pipe`
2019-12-10T22:53:33.643+05:30 [Error] PickRandom: Fail to find indexer for all index partitions. Num partition 1.  Partition with instances 0 
2019/12/10 22:53:33 Using n1ql client
2019/12/10 22:53:33 Using n1ql client
2019/12/10 22:53:35 []
2019/12/10 22:53:40 === Testing for persistence enabled = true, with interval = 10  ===
2019/12/10 22:53:40 DBG: ChangeIndexerSettings: nodes = [0xc009223c80]
2019/12/10 22:53:40 DBG: ChangeIndexerSettings: adminurl = 127.0.0.1:9106 host 127.0.0.1 sport 9106
2019/12/10 22:53:40 Changing config key indexer.statsPersistenceInterval to value 10
2019/12/10 22:53:40 []
2019-12-10T22:53:40.918+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:9106.  Error = EOF. Kill Pipe.
2019-12-10T22:53:40.918+05:30 [Error] WatcherServer.runOnce() : Watcher terminated unexpectedly.
2019-12-10T22:53:40.918+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:9106.  Error = EOF. Kill Pipe.
2019-12-10T22:53:40.919+05:30 [Error] WatcherServer.runOnce() : Watcher terminated unexpectedly.
2019-12-10T22:53:41.668+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2019-12-10T22:53:41.728+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:53:41.728+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:53:41.729+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2019-12-10T22:53:41.729+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2019-12-10T22:53:41.731+05:30 [Info] switched currmeta from 1660 -> 1661 force true 
2019-12-10T22:53:41.733+05:30 [Info] switched currmeta from 1665 -> 1665 force true 
2019/12/10 22:53:45 Using n1ql client
2019-12-10T22:53:45.896+05:30 [Info] GsiClient::UpdateUsecjson: using collatejson as data format between indexer and GsiClient
2019-12-10T22:53:45.897+05:30 [Error] transport error between 127.0.0.1:46888->127.0.0.1:9107: write tcp 127.0.0.1:46888->127.0.0.1:9107: write: broken pipe
2019-12-10T22:53:45.897+05:30 [Error] [GsiScanClient:"127.0.0.1:9107"] 5955775310500426855 request transport failed `write tcp 127.0.0.1:46888->127.0.0.1:9107: write: broken pipe`
2019-12-10T22:53:45.897+05:30 [Error] PickRandom: Fail to find indexer for all index partitions. Num partition 1.  Partition with instances 0 
2019/12/10 22:53:45 Using n1ql client
2019/12/10 22:53:45 Using n1ql client
2019/12/10 22:53:58 []
2019/12/10 22:54:03 === Testing for persistence enabled = true, with interval = 5  ===
2019/12/10 22:54:03 DBG: ChangeIndexerSettings: nodes = [0xc016b25c80]
2019/12/10 22:54:03 DBG: ChangeIndexerSettings: adminurl = 127.0.0.1:9106 host 127.0.0.1 sport 9106
2019/12/10 22:54:03 Changing config key indexer.statsPersistenceInterval to value 5
2019/12/10 22:54:03 []
2019/12/10 22:54:08 Using n1ql client
2019-12-10T22:54:08.242+05:30 [Error] transport error between 127.0.0.1:47190->127.0.0.1:9107: write tcp 127.0.0.1:47190->127.0.0.1:9107: write: broken pipe
2019-12-10T22:54:08.242+05:30 [Error] [GsiScanClient:"127.0.0.1:9107"] -8202892829527884110 request transport failed `write tcp 127.0.0.1:47190->127.0.0.1:9107: write: broken pipe`
2019-12-10T22:54:08.242+05:30 [Error] PickRandom: Fail to find indexer for all index partitions. Num partition 1.  Partition with instances 0 
2019/12/10 22:54:08 Using n1ql client
2019/12/10 22:54:08 Using n1ql client
2019/12/10 22:54:15 []
2019-12-10T22:54:15.393+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:9106.  Error = EOF. Kill Pipe.
2019-12-10T22:54:15.393+05:30 [Error] WatcherServer.runOnce() : Watcher terminated unexpectedly.
2019-12-10T22:54:15.393+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:9106.  Error = EOF. Kill Pipe.
2019-12-10T22:54:15.394+05:30 [Error] WatcherServer.runOnce() : Watcher terminated unexpectedly.
--- PASS: TestStatsPersistence (84.41s)
=== RUN   TestStats_StorageStatistics
2019/12/10 22:54:20 In TestStats_StorageStatistics()
2019/12/10 22:54:20 Index found:  index_age
2019/12/10 22:54:20 Stats from Index4 StorageStatistics for index index_age are []
2019/12/10 22:54:20 Index found:  p1
2019/12/10 22:54:20 Stats from Index4 StorageStatistics for index p1 are []
--- PASS: TestStats_StorageStatistics (0.00s)
FAIL
exit status 1
FAIL	github.com/couchbase/indexing/secondary/tests/functionaltests	7284.775s
Indexer Go routine dump logged in /opt/build/ns_server/logs/n_1/indexer_functests_pprof.log
curl: /opt/build/install/lib/libcurl.so.4: no version information available (required by curl)
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 51749    0 51749    0     0  8422k      0 --:--:-- --:--:-- --:--:--  9.8M
2019/12/10 22:54:23 In init()
2019/12/10 22:54:24 DBG: ChangeIndexerSettings: nodes = [0xc0001471c0]
2019/12/10 22:54:24 DBG: ChangeIndexerSettings: adminurl = 127.0.0.1:9106 host 127.0.0.1 sport 9106
2019/12/10 22:54:24 Changing config key indexer.api.enableTestServer to value true
2019/12/10 22:54:24 Using forestdb for creating indexes
2019/12/10 22:54:24 DBG: ChangeIndexerSettings: nodes = [0xc0003f8680]
2019/12/10 22:54:24 DBG: ChangeIndexerSettings: adminurl = 127.0.0.1:9106 host 127.0.0.1 sport 9106
2019/12/10 22:54:24 Changing config key indexer.settings.storage_mode to value forestdb
=== RUN   TestRangeWithConcurrentAddMuts
2019/12/10 22:54:29 In TestRangeWithConcurrentAddMuts()
2019/12/10 22:54:29 In DropAllSecondaryIndexes()
2019/12/10 22:54:29 Index found:  p1
2019/12/10 22:54:29 Dropped index p1
2019/12/10 22:54:29 Index found:  index_gender
2019/12/10 22:54:29 Dropped index index_gender
2019/12/10 22:54:29 Index found:  index_city
2019/12/10 22:54:29 Dropped index index_city
2019/12/10 22:54:29 Index found:  index_age
2019/12/10 22:54:29 Dropped index index_age
2019/12/10 22:54:29 Generating JSON docs
2019/12/10 22:54:29 Setting initial JSON docs in KV
2019/12/10 22:54:29 All indexers are active
2019/12/10 22:54:29 Creating a 2i
2019/12/10 22:54:32 Created the secondary index index_company. Waiting for it become active
2019/12/10 22:54:32 Index is now active
2019/12/10 22:54:32 In Range Scan for Thread 1: 
2019/12/10 22:54:32 CreateDocs:: Creating mutations
2019/12/10 22:54:33 ListAllSecondaryIndexes() for Thread 1: : Index index_company Bucket default
--- PASS: TestRangeWithConcurrentAddMuts (124.29s)
=== RUN   TestRangeWithConcurrentDelMuts
2019/12/10 22:56:33 In TestRangeWithConcurrentDelMuts()
2019/12/10 22:56:33 Generating JSON docs
2019-12-10T22:56:33.335+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:9106.  Error = read tcp 127.0.0.1:48092->127.0.0.1:9106: use of closed network connection. Kill Pipe.
2019/12/10 22:56:35 Setting initial JSON docs in KV
2019/12/10 22:56:51 All indexers are active
2019/12/10 22:56:51 Creating a 2i
2019/12/10 22:56:51 Index found:  index_company
2019/12/10 22:56:51 In Range Scan for Thread 1: 
2019/12/10 22:56:51 CreateDocs:: Delete mutations
2019/12/10 22:56:52 ListAllSecondaryIndexes() for Thread 1: : Index index_company Bucket default
--- PASS: TestRangeWithConcurrentDelMuts (139.14s)
=== RUN   TestScanWithConcurrentIndexOps
2019/12/10 22:58:52 In TestScanWithConcurrentIndexOps()
2019-12-10T22:58:52.480+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:9106.  Error = read tcp 127.0.0.1:50404->127.0.0.1:9106: use of closed network connection. Kill Pipe.
2019/12/10 22:58:52 Generating JSON docs
2019/12/10 22:58:58 Setting initial JSON docs in KV
2019/12/10 22:59:51 All indexers are active
2019/12/10 22:59:51 Creating a 2i
2019/12/10 22:59:51 Index found:  index_company
2019/12/10 22:59:51 In Range Scan for Thread 1: 
2019/12/10 22:59:51 Create and Drop index operations
2019/12/10 22:59:51 ListAllSecondaryIndexes() for Thread 1: : Index index_company Bucket default
2019/12/10 23:00:25 Created the secondary index index_age. Waiting for it become active
2019/12/10 23:00:25 Index is now active
2019/12/10 23:00:58 Created the secondary index index_firstname. Waiting for it become active
2019/12/10 23:00:58 Index is now active
2019/12/10 23:01:02 Dropping the secondary index index_age
2019/12/10 23:01:02 Index dropped
2019/12/10 23:01:03 Dropping the secondary index index_firstname
2019/12/10 23:01:03 Index dropped
2019/12/10 23:01:36 Created the secondary index index_age. Waiting for it become active
2019/12/10 23:01:36 Index is now active
2019-12-10T23:01:52.767+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:9106.  Error = read tcp 127.0.0.1:58830->127.0.0.1:9106: use of closed network connection. Kill Pipe.
2019/12/10 23:02:07 Created the secondary index index_firstname. Waiting for it become active
2019/12/10 23:02:07 Index is now active
2019/12/10 23:02:10 Dropping the secondary index index_age
2019/12/10 23:02:10 Index dropped
2019/12/10 23:02:11 Dropping the secondary index index_firstname
2019/12/10 23:02:11 Index dropped
--- PASS: TestScanWithConcurrentIndexOps (200.43s)
=== RUN   TestConcurrentScans_SameIndex
2019/12/10 23:02:12 In TestConcurrentScans_SameIndex()
2019/12/10 23:02:12 Generating JSON docs
2019-12-10T23:02:12.908+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:9106.  Error = read tcp 127.0.0.1:58828->127.0.0.1:9106: use of closed network connection. Kill Pipe.
2019/12/10 23:02:18 Setting initial JSON docs in KV
2019/12/10 23:03:12 All indexers are active
2019/12/10 23:03:12 Creating a 2i
2019/12/10 23:03:12 Index found:  index_company
2019/12/10 23:03:12 In Range Scan for Thread 6: 
2019/12/10 23:03:12 In Range Scan for Thread 3: 
2019/12/10 23:03:12 In Range Scan for Thread 1: 
2019/12/10 23:03:12 In Range Scan for Thread 4: 
2019/12/10 23:03:12 In Range Scan for Thread 2: 
2019/12/10 23:03:12 In Range Scan for Thread 5: 
2019/12/10 23:03:12 ListAllSecondaryIndexes() for Thread 3: : Index index_company Bucket default
2019/12/10 23:03:12 ListAllSecondaryIndexes() for Thread 5: : Index index_company Bucket default
2019/12/10 23:03:12 ListAllSecondaryIndexes() for Thread 6: : Index index_company Bucket default
2019/12/10 23:03:12 ListAllSecondaryIndexes() for Thread 1: : Index index_company Bucket default
2019/12/10 23:03:13 ListAllSecondaryIndexes() for Thread 2: : Index index_company Bucket default
2019/12/10 23:03:13 ListAllSecondaryIndexes() for Thread 4: : Index index_company Bucket default
2019-12-10T23:05:13.464+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:9106.  Error = read tcp 127.0.0.1:59336->127.0.0.1:9106: use of closed network connection. Kill Pipe.
2019-12-10T23:05:14.369+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:9106.  Error = read tcp 127.0.0.1:59332->127.0.0.1:9106: use of closed network connection. Kill Pipe.
2019-12-10T23:05:16.347+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:9106.  Error = read tcp 127.0.0.1:59344->127.0.0.1:9106: use of closed network connection. Kill Pipe.
2019-12-10T23:05:17.265+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:9106.  Error = read tcp 127.0.0.1:59342->127.0.0.1:9106: use of closed network connection. Kill Pipe.
2019-12-10T23:05:19.711+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:9106.  Error = read tcp 127.0.0.1:59338->127.0.0.1:9106: use of closed network connection. Kill Pipe.
--- PASS: TestConcurrentScans_SameIndex (187.41s)
=== RUN   TestConcurrentScans_MultipleIndexes
2019/12/10 23:05:20 In TestConcurrentScans_MultipleIndexes()
2019/12/10 23:05:20 Generating JSON docs
2019-12-10T23:05:20.318+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:9106.  Error = read tcp 127.0.0.1:59340->127.0.0.1:9106: use of closed network connection. Kill Pipe.
2019/12/10 23:05:26 Setting initial JSON docs in KV
2019/12/10 23:06:20 All indexers are active
2019/12/10 23:06:20 Creating multiple indexes
2019/12/10 23:06:20 Index found:  index_company
2019/12/10 23:07:11 Created the secondary index index_age. Waiting for it become active
2019/12/10 23:07:11 Index is now active
2019/12/10 23:07:59 Created the secondary index index_firstname. Waiting for it become active
2019/12/10 23:07:59 Index is now active
2019/12/10 23:07:59 In Range Scan for Thread 3: 
2019/12/10 23:07:59 In Range Scan for Thread 1: 
2019/12/10 23:07:59 In Range Scan
2019/12/10 23:07:59 ListAllSecondaryIndexes() for Thread 3: : Index index_age Bucket default
2019/12/10 23:07:59 ListAllSecondaryIndexes() for Thread 3: : Index index_company Bucket default
2019/12/10 23:07:59 ListAllSecondaryIndexes() for Thread 3: : Index index_firstname Bucket default
2019/12/10 23:08:00 ListAllSecondaryIndexes() for Thread 1: : Index index_age Bucket default
2019/12/10 23:08:00 ListAllSecondaryIndexes() for Thread 1: : Index index_company Bucket default
2019/12/10 23:08:00 ListAllSecondaryIndexes() for Thread 1: : Index index_firstname Bucket default
2019-12-10T23:10:00.389+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:9106.  Error = read tcp 127.0.0.1:60254->127.0.0.1:9106: use of closed network connection. Kill Pipe.
2019-12-10T23:10:00.713+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:9106.  Error = read tcp 127.0.0.1:60258->127.0.0.1:9106: use of closed network connection. Kill Pipe.
--- PASS: TestConcurrentScans_MultipleIndexes (282.85s)
=== RUN   TestMutationsWithMultipleIndexBuilds
2019/12/10 23:10:03 In TestMutationsWithMultipleIndexBuilds()
2019/12/10 23:10:03 In DropAllSecondaryIndexes()
2019-12-10T23:10:03.171+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:9106.  Error = read tcp 127.0.0.1:60256->127.0.0.1:9106: use of closed network connection. Kill Pipe.
2019/12/10 23:10:03 Index found:  index_firstname
2019/12/10 23:10:03 Dropped index index_firstname
2019/12/10 23:10:03 Index found:  index_company
2019/12/10 23:10:03 Dropped index index_company
2019/12/10 23:10:03 Index found:  index_age
2019/12/10 23:10:03 Dropped index index_age
2019/12/10 23:10:03 Generating JSON docs
2019/12/10 23:10:09 Setting initial JSON docs in KV
2019/12/10 23:11:25 Created the secondary index index_primary. Waiting for it become active
2019/12/10 23:11:25 Index is now active
2019/12/10 23:11:25 Creating multiple indexes in deferred mode
2019/12/10 23:11:26 Created the secondary index index_company
2019/12/10 23:11:26 Created the secondary index index_age
2019/12/10 23:11:27 Created the secondary index index_firstname
2019/12/10 23:11:27 Created the secondary index index_lastname
2019/12/10 23:11:27 Build Indexes and wait for indexes to become active: [index_company index_age index_firstname index_lastname]
2019/12/10 23:11:27 Build command issued for the deferred indexes [index_company index_age index_firstname index_lastname]
2019/12/10 23:11:27 Waiting for the index index_company to become active
2019/12/10 23:11:27 Waiting for index to go active ...
2019/12/10 23:11:28 Waiting for index to go active ...
2019/12/10 23:11:29 Waiting for index to go active ...
2019/12/10 23:11:30 Waiting for index to go active ...
2019/12/10 23:11:31 Waiting for index to go active ...
2019/12/10 23:11:32 Waiting for index to go active ...
2019/12/10 23:11:33 Waiting for index to go active ...
2019/12/10 23:11:35 Waiting for index to go active ...
2019/12/10 23:11:36 Waiting for index to go active ...
2019/12/10 23:11:37 Waiting for index to go active ...
2019/12/10 23:11:38 Waiting for index to go active ...
2019/12/10 23:11:39 Waiting for index to go active ...
2019/12/10 23:11:40 Waiting for index to go active ...
2019/12/10 23:11:41 Waiting for index to go active ...
2019/12/10 23:11:42 Waiting for index to go active ...
2019/12/10 23:11:43 Waiting for index to go active ...
2019/12/10 23:11:44 Waiting for index to go active ...
2019/12/10 23:11:45 Waiting for index to go active ...
2019/12/10 23:11:46 Waiting for index to go active ...
2019/12/10 23:11:47 Waiting for index to go active ...
2019/12/10 23:11:48 Waiting for index to go active ...
2019/12/10 23:11:49 Waiting for index to go active ...
2019/12/10 23:11:50 Waiting for index to go active ...
2019/12/10 23:11:51 Waiting for index to go active ...
2019/12/10 23:11:52 Waiting for index to go active ...
2019/12/10 23:11:53 Waiting for index to go active ...
2019/12/10 23:11:54 Waiting for index to go active ...
2019/12/10 23:11:55 Waiting for index to go active ...
2019/12/10 23:11:56 Waiting for index to go active ...
2019/12/10 23:11:57 Waiting for index to go active ...
2019/12/10 23:11:58 Waiting for index to go active ...
2019/12/10 23:11:59 Waiting for index to go active ...
2019/12/10 23:12:00 Waiting for index to go active ...
2019/12/10 23:12:01 Waiting for index to go active ...
2019/12/10 23:12:02 Waiting for index to go active ...
2019/12/10 23:12:03 Waiting for index to go active ...
2019/12/10 23:12:04 Waiting for index to go active ...
2019/12/10 23:12:05 Waiting for index to go active ...
2019/12/10 23:12:06 Waiting for index to go active ...
2019/12/10 23:12:07 Waiting for index to go active ...
2019/12/10 23:12:08 Waiting for index to go active ...
2019/12/10 23:12:09 Waiting for index to go active ...
2019/12/10 23:12:10 Waiting for index to go active ...
2019/12/10 23:12:11 Waiting for index to go active ...
2019/12/10 23:12:12 Waiting for index to go active ...
2019/12/10 23:12:13 Waiting for index to go active ...
2019/12/10 23:12:14 Waiting for index to go active ...
2019/12/10 23:12:15 Waiting for index to go active ...
2019/12/10 23:12:16 Waiting for index to go active ...
2019/12/10 23:12:17 Waiting for index to go active ...
2019/12/10 23:12:18 Waiting for index to go active ...
2019/12/10 23:12:19 Waiting for index to go active ...
2019/12/10 23:12:20 Waiting for index to go active ...
2019/12/10 23:12:21 Waiting for index to go active ...
2019/12/10 23:12:22 Waiting for index to go active ...
2019/12/10 23:12:23 Waiting for index to go active ...
2019/12/10 23:12:24 Waiting for index to go active ...
2019/12/10 23:12:25 Waiting for index to go active ...
2019/12/10 23:12:26 Waiting for index to go active ...
2019/12/10 23:12:27 Waiting for index to go active ...
2019/12/10 23:12:28 Waiting for index to go active ...
2019/12/10 23:12:29 Waiting for index to go active ...
2019/12/10 23:12:30 Waiting for index to go active ...
2019/12/10 23:12:31 Waiting for index to go active ...
2019/12/10 23:12:32 Waiting for index to go active ...
2019/12/10 23:12:33 Waiting for index to go active ...
2019/12/10 23:12:34 Waiting for index to go active ...
2019/12/10 23:12:35 Waiting for index to go active ...
2019/12/10 23:12:36 Waiting for index to go active ...
2019/12/10 23:12:37 Waiting for index to go active ...
2019/12/10 23:12:38 Waiting for index to go active ...
2019/12/10 23:12:39 Waiting for index to go active ...
2019/12/10 23:12:40 Waiting for index to go active ...
2019/12/10 23:12:41 Waiting for index to go active ...
2019/12/10 23:12:42 Waiting for index to go active ...
2019/12/10 23:12:43 Waiting for index to go active ...
2019/12/10 23:12:44 Waiting for index to go active ...
2019/12/10 23:12:45 Waiting for index to go active ...
2019/12/10 23:12:46 Waiting for index to go active ...
2019/12/10 23:12:47 Waiting for index to go active ...
2019/12/10 23:12:48 Waiting for index to go active ...
2019/12/10 23:12:49 Waiting for index to go active ...
2019/12/10 23:12:50 Waiting for index to go active ...
2019/12/10 23:12:51 Waiting for index to go active ...
2019/12/10 23:12:52 Waiting for index to go active ...
2019/12/10 23:12:53 Waiting for index to go active ...
2019/12/10 23:12:54 Waiting for index to go active ...
2019/12/10 23:12:55 Waiting for index to go active ...
2019/12/10 23:12:56 Waiting for index to go active ...
2019/12/10 23:12:57 Waiting for index to go active ...
2019/12/10 23:12:58 Waiting for index to go active ...
2019/12/10 23:12:59 Waiting for index to go active ...
2019/12/10 23:13:00 Waiting for index to go active ...
2019/12/10 23:13:01 Waiting for index to go active ...
2019/12/10 23:13:02 Waiting for index to go active ...
2019/12/10 23:13:03 Waiting for index to go active ...
2019/12/10 23:13:04 Waiting for index to go active ...
2019/12/10 23:13:05 Waiting for index to go active ...
2019/12/10 23:13:06 Waiting for index to go active ...
2019/12/10 23:13:07 Waiting for index to go active ...
2019/12/10 23:13:08 Waiting for index to go active ...
2019/12/10 23:13:09 Waiting for index to go active ...
2019/12/10 23:13:10 Waiting for index to go active ...
2019/12/10 23:13:11 Waiting for index to go active ...
2019/12/10 23:13:12 Waiting for index to go active ...
2019/12/10 23:13:13 Waiting for index to go active ...
2019/12/10 23:13:14 Waiting for index to go active ...
2019/12/10 23:13:15 Waiting for index to go active ...
2019/12/10 23:13:16 Waiting for index to go active ...
2019/12/10 23:13:17 Index is now active
2019/12/10 23:13:17 Waiting for the index index_age to become active
2019/12/10 23:13:17 Index is now active
2019/12/10 23:13:17 Waiting for the index index_firstname to become active
2019/12/10 23:13:17 Index is now active
2019/12/10 23:13:17 Waiting for the index index_lastname to become active
2019/12/10 23:13:17 Index is now active
--- PASS: TestMutationsWithMultipleIndexBuilds (193.87s)
PASS
ok  	github.com/couchbase/indexing/secondary/tests/largedatatests	1133.275s
Indexer Go routine dump logged in /opt/build/ns_server/logs/n_1/indexer_largedata_pprof.log
curl: /opt/build/install/lib/libcurl.so.4: no version information available (required by curl)
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 51525    0 51525    0     0   9.8M      0 --:--:-- --:--:-- --:--:--  9.8M

Integration tests

echo "Running 2i integration tests with 4 node cluster"
Running 2i integration tests with 4 node cluster
scripts/start_cluster_and_run_tests.sh b/resources/dev-4-nodes-xdcr_n1ql_2i.ini conf/simple_2i_n1ql.conf 1 1 gsi_type=forestdb
/opt/build/testrunner /opt/build/testrunner
make[1]: Entering directory '/opt/build/ns_server'
cd build && make --no-print-directory ns_dataclean
Built target ns_dataclean
make[1]: Leaving directory '/opt/build/ns_server'
make[1]: Entering directory '/opt/build/ns_server'
cd build && make --no-print-directory all
==> ale (compile)
Built target ale
Built target ns_cfg
==> ns_server (compile)
Built target ns_server
Building Go target vbmap using Go 1.8.5
Built target vbmap
Building Go target ns_gozip using Go 1.8.5
Built target ns_gozip
Building Go target ns_goport using Go 1.8.5
Built target ns_goport
Building Go target ns_generate_cert using Go 1.11.6
Built target ns_generate_cert
Building Go target ns_godu using Go 1.8.5
Built target ns_godu
Building Go target ns_minify using Go 1.8.5
Built target ns_minify
-- Building app.min.js and index.min.html ...
Built target minify
==> gen_smtp (compile)
Built target gen_smtp
==> ns_babysitter (compile)
Built target ns_babysitter
==> ns_couchdb (compile)
Built target ns_couchdb
Building Go target ns_gosecrets using Go 1.8.5
Built target ns_gosecrets
make[1]: Leaving directory '/opt/build/ns_server'
/opt/build/testrunner
filename: conf/simple_2i_n1ql.conf
prefix: 2i.indexscans_2i.SecondaryIndexingScanTests
prefix: 2i.indexcreatedrop_2i.SecondaryIndexingCreateDropTests
prefix: 2i.cluster_ops_2i.SecondaryIndexingClusterOpsTests
Global Test input params:
{'cluster_name': 'dev-4-nodes-xdcr_n1ql_2i',
 'conf_file': 'conf/simple_2i_n1ql.conf',
 'gsi_type': 'forestdb',
 'ini': 'b/resources/dev-4-nodes-xdcr_n1ql_2i.ini',
 'makefile': 'True',
 'num_nodes': 4,
 'spec': 'simple_2i_n1ql'}
Logs will be stored at /opt/build/testrunner/logs/testrunner-19-Dec-10_23-14-02/test_1

./testrunner -i b/resources/dev-4-nodes-xdcr_n1ql_2i.ini -p makefile=True -t 2i.indexscans_2i.SecondaryIndexingScanTests.test_multi_create_query_explain_drop_index,groups=simple:equals:no_orderby_groupby:range,dataset=default,doc-per-day=1,use_gsi_for_primary=True,use_gsi_for_secondary=True,GROUP=gsi

Test Input params:
{'doc-per-day': '1', 'use_gsi_for_primary': 'True', 'GROUP': 'gsi', 'use_gsi_for_secondary': 'True', 'gsi_type': 'forestdb', 'makefile': 'True', 'conf_file': 'conf/simple_2i_n1ql.conf', 'dataset': 'default', 'cluster_name': 'dev-4-nodes-xdcr_n1ql_2i', 'ini': 'b/resources/dev-4-nodes-xdcr_n1ql_2i.ini', 'groups': 'simple:equals:no_orderby_groupby:range', 'case_number': 1, 'num_nodes': 4, 'logs_folder': '/opt/build/testrunner/logs/testrunner-19-Dec-10_23-14-02/test_1', 'spec': 'simple_2i_n1ql'}
Run before suite setup for 2i.indexscans_2i.SecondaryIndexingScanTests.test_multi_create_query_explain_drop_index
2019-12-10 23:14:03 | WARNING | MainProcess | MainThread | [data_helper.] Can not import concurrent module. Data for each server will be loaded/retrieved sequentially
test_multi_create_query_explain_drop_index (2i.indexscans_2i.SecondaryIndexingScanTests) ... 2019-12-10 23:14:03 | INFO | MainProcess | test_thread | [remote_util.__init__] connecting to 127.0.0.1 with username:Administrator 
2019-12-10 23:14:03 | INFO | MainProcess | test_thread | [remote_util.__init__] Connected to 127.0.0.1
2019-12-10 23:14:03 | ERROR | MainProcess | test_thread | [rest_client._http_request] socket error while connecting to http://127.0.0.1:9000/nodes/self error [Errno 111] Connection refused 
2019-12-10 23:14:06 | ERROR | MainProcess | test_thread | [rest_client._http_request] socket error while connecting to http://127.0.0.1:9000/nodes/self error [Errno 111] Connection refused 
2019-12-10 23:14:09 | ERROR | MainProcess | test_thread | [rest_client._http_request] socket error while connecting to http://127.0.0.1:9000/nodes/self error [Errno 111] Connection refused 
2019-12-10 23:14:15 | INFO | MainProcess | test_thread | [rest_client.get_nodes_version] Node version in cluster 6.5.0-4928-rel-enterprise
2019-12-10 23:14:15 | ERROR | MainProcess | test_thread | [rest_client._http_request] GET http://127.0.0.1:9000/pools/default body:  headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 404 reason: unknown "unknown pool" auth: Administrator:asdasd
2019-12-10 23:14:15 | INFO | MainProcess | test_thread | [rest_client.get_nodes_versions] Node versions in cluster []
2019-12-10 23:14:15 | INFO | MainProcess | test_thread | [basetestcase.setUp] ==============  basetestcase setup was started for test #1 test_multi_create_query_explain_drop_index==============
2019-12-10 23:14:15 | ERROR | MainProcess | test_thread | [rest_client._http_request] GET http://127.0.0.1:9000/pools/default/ body:  headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 404 reason: unknown "unknown pool" auth: Administrator:asdasd
2019-12-10 23:14:16 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] waiting for ns_server @ 127.0.0.1:9000
2019-12-10 23:14:16 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] ns_server @ 127.0.0.1:9000 is running
2019-12-10 23:14:16 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] waiting for ns_server @ 127.0.0.1:9001
2019-12-10 23:14:16 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] ns_server @ 127.0.0.1:9001 is running
2019-12-10 23:14:16 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] waiting for ns_server @ 127.0.0.1:9002
2019-12-10 23:14:16 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] ns_server @ 127.0.0.1:9002 is running
2019-12-10 23:14:16 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] waiting for ns_server @ 127.0.0.1:9003
2019-12-10 23:14:16 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] ns_server @ 127.0.0.1:9003 is running
2019-12-10 23:14:16 | ERROR | MainProcess | test_thread | [rest_client._http_request] GET http://127.0.0.1:9000/pools/default body:  headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 404 reason: unknown "unknown pool" auth: Administrator:asdasd
2019-12-10 23:14:16 | ERROR | MainProcess | test_thread | [rest_client._http_request] GET http://127.0.0.1:9000/pools/default body:  headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 404 reason: unknown "unknown pool" auth: Administrator:asdasd
2019-12-10 23:14:16 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] cannot find service node index in cluster 
2019-12-10 23:14:16 | ERROR | MainProcess | test_thread | [rest_client._http_request] GET http://127.0.0.1:9000/pools/default body:  headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 404 reason: unknown "unknown pool" auth: Administrator:asdasd
2019-12-10 23:14:16 | INFO | MainProcess | test_thread | [basetestcase.print_cluster_stats] ------- Cluster statistics -------
2019-12-10 23:14:16 | INFO | MainProcess | test_thread | [basetestcase.print_cluster_stats] --- End of cluster statistics ---
2019-12-10 23:14:16 | INFO | MainProcess | test_thread | [basetestcase.tearDown] ==============  basetestcase cleanup was started for test #1 test_multi_create_query_explain_drop_index ==============
2019-12-10 23:14:16 | ERROR | MainProcess | test_thread | [rest_client._http_request] GET http://127.0.0.1:9000/pools/default/ body:  headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 404 reason: unknown "unknown pool" auth: Administrator:asdasd
2019-12-10 23:14:16 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] waiting for ns_server @ 127.0.0.1:9000
2019-12-10 23:14:16 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] ns_server @ 127.0.0.1:9000 is running
2019-12-10 23:14:16 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] waiting for ns_server @ 127.0.0.1:9001
2019-12-10 23:14:16 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] ns_server @ 127.0.0.1:9001 is running
2019-12-10 23:14:16 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] waiting for ns_server @ 127.0.0.1:9002
2019-12-10 23:14:16 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] ns_server @ 127.0.0.1:9002 is running
2019-12-10 23:14:16 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] waiting for ns_server @ 127.0.0.1:9003
2019-12-10 23:14:16 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] ns_server @ 127.0.0.1:9003 is running
2019-12-10 23:14:16 | INFO | MainProcess | test_thread | [basetestcase.tearDown] ==============  basetestcase cleanup was finished for test #1 test_multi_create_query_explain_drop_index ==============
Cluster instance shutdown with force
2019-12-10 23:14:16 | INFO | MainProcess | test_thread | [basetestcase.setUp] initializing cluster
2019-12-10 23:14:20 | INFO | MainProcess | Cluster_Thread | [task.execute] server: ip:127.0.0.1 port:9000 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_0@cb.local', 'uptime': u'14', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9000', 'storage': [], 'moxi': 11211, 'port': u'9000', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12000, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 14005035008, 'memoryTotal': 15466930176, 'memoryQuota': 8184, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
2019-12-10 23:14:20 | ERROR | MainProcess | Cluster_Thread | [rest_client._http_request] GET http://127.0.0.1:9000/pools/default body:  headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 404 reason: unknown "unknown pool" auth: Administrator:asdasd
2019-12-10 23:14:20 | INFO | MainProcess | Cluster_Thread | [task.execute] quota for index service will be 256 MB
2019-12-10 23:14:20 | INFO | MainProcess | Cluster_Thread | [task.execute] set index quota to node 127.0.0.1 
2019-12-10 23:14:20 | INFO | MainProcess | Cluster_Thread | [rest_client.set_service_memoryQuota] pools/default params : indexMemoryQuota=256
2019-12-10 23:14:20 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster_memoryQuota] pools/default params : memoryQuota=7650
2019-12-10 23:14:20 | INFO | MainProcess | Cluster_Thread | [rest_client.init_node_services] /node/controller/setupServices params on 127.0.0.1:9000:services=kv%2Cindex%2Cn1ql&password=asdasd&user=Administrator&hostname=127.0.0.1%3A9000
2019-12-10 23:14:20 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster] settings/web params on 127.0.0.1:9000:username=Administrator&password=asdasd&port=9000
2019-12-10 23:14:20 | INFO | MainProcess | Cluster_Thread | [remote_util.__init__] connecting to 127.0.0.1 with username:Administrator 
2019-12-10 23:14:20 | INFO | MainProcess | Cluster_Thread | [remote_util.__init__] Connected to 127.0.0.1
2019-12-10 23:14:20 | INFO | MainProcess | Cluster_Thread | [remote_util.execute_command_raw] running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9000/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
2019-12-10 23:14:20 | INFO | MainProcess | Cluster_Thread | [remote_util.execute_command_raw] command executed successfully
2019-12-10 23:14:20 | INFO | MainProcess | Cluster_Thread | [rest_client.diag_eval] /diag/eval status on 127.0.0.1:9000: True content: [6,5] command: cluster_compat_mode:get_compat_version().
2019-12-10 23:14:20 | INFO | MainProcess | Cluster_Thread | [rest_client.set_indexer_storage_mode] settings/indexes params : storageMode=forestdb
2019-12-10 23:14:20 | ERROR | MainProcess | Cluster_Thread | [rest_client._http_request] POST http://127.0.0.1:9000/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
2019-12-10 23:14:20 | INFO | MainProcess | Cluster_Thread | [task.execute] server: ip:127.0.0.1 port:9001 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_1@cb.local', 'uptime': u'14', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9001', 'storage': [], 'moxi': 11211, 'port': u'9001', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12002, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 14007595008, 'memoryTotal': 15466930176, 'memoryQuota': 8184, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
2019-12-10 23:14:20 | ERROR | MainProcess | Cluster_Thread | [rest_client._http_request] GET http://127.0.0.1:9001/pools/default body:  headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 404 reason: unknown "unknown pool" auth: Administrator:asdasd
2019-12-10 23:14:20 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster_memoryQuota] pools/default params : memoryQuota=7906
2019-12-10 23:14:20 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster] settings/web params on 127.0.0.1:9001:username=Administrator&password=asdasd&port=9001
2019-12-10 23:14:21 | INFO | MainProcess | Cluster_Thread | [remote_util.__init__] connecting to 127.0.0.1 with username:Administrator 
2019-12-10 23:14:21 | INFO | MainProcess | Cluster_Thread | [remote_util.__init__] Connected to 127.0.0.1
2019-12-10 23:14:21 | INFO | MainProcess | Cluster_Thread | [remote_util.execute_command_raw] running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9001/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
2019-12-10 23:14:21 | INFO | MainProcess | Cluster_Thread | [remote_util.execute_command_raw] command executed successfully
2019-12-10 23:14:21 | INFO | MainProcess | Cluster_Thread | [rest_client.diag_eval] /diag/eval status on 127.0.0.1:9001: True content: [6,5] command: cluster_compat_mode:get_compat_version().
2019-12-10 23:14:21 | INFO | MainProcess | Cluster_Thread | [rest_client.set_indexer_storage_mode] settings/indexes params : storageMode=forestdb
2019-12-10 23:14:21 | ERROR | MainProcess | Cluster_Thread | [rest_client._http_request] POST http://127.0.0.1:9001/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
2019-12-10 23:14:21 | INFO | MainProcess | Cluster_Thread | [task.execute] server: ip:127.0.0.1 port:9002 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_2@cb.local', 'uptime': u'14', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9002', 'storage': [], 'moxi': 11211, 'port': u'9002', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12004, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 14008999936, 'memoryTotal': 15466930176, 'memoryQuota': 8184, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
2019-12-10 23:14:21 | ERROR | MainProcess | Cluster_Thread | [rest_client._http_request] GET http://127.0.0.1:9002/pools/default body:  headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 404 reason: unknown "unknown pool" auth: Administrator:asdasd
2019-12-10 23:14:21 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster_memoryQuota] pools/default params : memoryQuota=7906
2019-12-10 23:14:21 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster] settings/web params on 127.0.0.1:9002:username=Administrator&password=asdasd&port=9002
2019-12-10 23:14:21 | INFO | MainProcess | Cluster_Thread | [remote_util.__init__] connecting to 127.0.0.1 with username:Administrator 
2019-12-10 23:14:21 | INFO | MainProcess | Cluster_Thread | [remote_util.__init__] Connected to 127.0.0.1
2019-12-10 23:14:21 | INFO | MainProcess | Cluster_Thread | [remote_util.execute_command_raw] running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9002/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
2019-12-10 23:14:21 | INFO | MainProcess | Cluster_Thread | [remote_util.execute_command_raw] command executed successfully
2019-12-10 23:14:21 | INFO | MainProcess | Cluster_Thread | [rest_client.diag_eval] /diag/eval status on 127.0.0.1:9002: True content: [6,5] command: cluster_compat_mode:get_compat_version().
2019-12-10 23:14:21 | INFO | MainProcess | Cluster_Thread | [rest_client.set_indexer_storage_mode] settings/indexes params : storageMode=forestdb
2019-12-10 23:14:21 | ERROR | MainProcess | Cluster_Thread | [rest_client._http_request] POST http://127.0.0.1:9002/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
2019-12-10 23:14:21 | INFO | MainProcess | Cluster_Thread | [task.execute] server: ip:127.0.0.1 port:9003 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_3@cb.local', 'uptime': u'14', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9003', 'storage': [], 'moxi': 11211, 'port': u'9003', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12006, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 14011162624, 'memoryTotal': 15466930176, 'memoryQuota': 8184, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
2019-12-10 23:14:21 | ERROR | MainProcess | Cluster_Thread | [rest_client._http_request] GET http://127.0.0.1:9003/pools/default body:  headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 404 reason: unknown "unknown pool" auth: Administrator:asdasd
2019-12-10 23:14:21 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster_memoryQuota] pools/default params : memoryQuota=7906
2019-12-10 23:14:21 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster] settings/web params on 127.0.0.1:9003:username=Administrator&password=asdasd&port=9003
2019-12-10 23:14:21 | INFO | MainProcess | Cluster_Thread | [remote_util.__init__] connecting to 127.0.0.1 with username:Administrator 
2019-12-10 23:14:21 | INFO | MainProcess | Cluster_Thread | [remote_util.__init__] Connected to 127.0.0.1
2019-12-10 23:14:22 | INFO | MainProcess | Cluster_Thread | [remote_util.execute_command_raw] running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9003/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
2019-12-10 23:14:22 | INFO | MainProcess | Cluster_Thread | [remote_util.execute_command_raw] command executed successfully
2019-12-10 23:14:22 | INFO | MainProcess | Cluster_Thread | [rest_client.diag_eval] /diag/eval status on 127.0.0.1:9003: True content: [6,5] command: cluster_compat_mode:get_compat_version().
2019-12-10 23:14:22 | INFO | MainProcess | Cluster_Thread | [rest_client.set_indexer_storage_mode] settings/indexes params : storageMode=forestdb
2019-12-10 23:14:22 | ERROR | MainProcess | Cluster_Thread | [rest_client._http_request] POST http://127.0.0.1:9003/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
2019-12-10 23:14:22 | INFO | MainProcess | test_thread | [basetestcase.add_built_in_server_user] **** add built-in 'cbadminbucket' user to node 127.0.0.1 ****
2019-12-10 23:14:22 | ERROR | MainProcess | test_thread | [rest_client._http_request] DELETE http://127.0.0.1:9000/settings/rbac/users/local/cbadminbucket body:  headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 404 reason: unknown "User was not found." auth: Administrator:asdasd
2019-12-10 23:14:22 | INFO | MainProcess | test_thread | [internal_user.delete_user] Exception while deleting user. Exception is -"User was not found."
2019-12-10 23:14:22 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 5 secs.  ...
2019-12-10 23:14:27 | INFO | MainProcess | test_thread | [basetestcase.add_built_in_server_user] **** add 'admin' role to 'cbadminbucket' user ****
2019-12-10 23:14:27 | INFO | MainProcess | test_thread | [basetestcase.setUp] done initializing cluster
2019-12-10 23:14:27 | INFO | MainProcess | test_thread | [remote_util.__init__] connecting to 127.0.0.1 with username:Administrator 
2019-12-10 23:14:27 | INFO | MainProcess | test_thread | [remote_util.__init__] Connected to 127.0.0.1
2019-12-10 23:14:27 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9000/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
2019-12-10 23:14:27 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully
2019-12-10 23:14:27 | INFO | MainProcess | test_thread | [basetestcase.enable_diag_eval_on_non_local_hosts] Enabled diag/eval for non-local hosts from 127.0.0.1
2019-12-10 23:14:28 | INFO | MainProcess | Cluster_Thread | [rest_client.create_bucket] http://127.0.0.1:9000/pools/default/buckets with param: bucketType=membase&evictionPolicy=valueOnly&threadsNumber=3&ramQuotaMB=7650&authType=sasl&name=default&flushEnabled=1&replicaNumber=1&compressionMode=passive&saslPassword=None&replicaIndex=1
2019-12-10 23:14:28 | INFO | MainProcess | Cluster_Thread | [rest_client.create_bucket] 0.01 seconds to create bucket default
2019-12-10 23:14:28 | INFO | MainProcess | Cluster_Thread | [bucket_helper.wait_for_memcached] waiting for memcached bucket : default in 127.0.0.1 to accept set ops
2019-12-10 23:14:28 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 127.0.0.1:12000 default
2019-12-10 23:14:28 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 127.0.0.1:12000 default
2019-12-10 23:14:28 | WARNING | MainProcess | Cluster_Thread | [task.check] vbucket map not ready after try 0
2019-12-10 23:14:28 | INFO | MainProcess | Cluster_Thread | [bucket_helper.wait_for_memcached] waiting for memcached bucket : default in 127.0.0.1 to accept set ops
2019-12-10 23:14:28 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 127.0.0.1:12000 default
2019-12-10 23:14:28 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 127.0.0.1:12000 default
2019-12-10 23:14:28 | WARNING | MainProcess | Cluster_Thread | [task.check] vbucket map not ready after try 1
2019-12-10 23:14:28 | INFO | MainProcess | Cluster_Thread | [bucket_helper.wait_for_memcached] waiting for memcached bucket : default in 127.0.0.1 to accept set ops
2019-12-10 23:14:28 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 127.0.0.1:12000 default
2019-12-10 23:14:28 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 127.0.0.1:12000 default
2019-12-10 23:14:28 | WARNING | MainProcess | Cluster_Thread | [task.check] vbucket map not ready after try 2
2019-12-10 23:14:28 | INFO | MainProcess | Cluster_Thread | [bucket_helper.wait_for_memcached] waiting for memcached bucket : default in 127.0.0.1 to accept set ops
2019-12-10 23:14:29 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 127.0.0.1:12000 default
2019-12-10 23:14:29 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 127.0.0.1:12000 default
2019-12-10 23:14:29 | WARNING | MainProcess | Cluster_Thread | [task.check] vbucket map not ready after try 3
2019-12-10 23:14:29 | INFO | MainProcess | Cluster_Thread | [bucket_helper.wait_for_memcached] waiting for memcached bucket : default in 127.0.0.1 to accept set ops
2019-12-10 23:14:29 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 127.0.0.1:12000 default
2019-12-10 23:14:29 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 127.0.0.1:12000 default
2019-12-10 23:14:29 | WARNING | MainProcess | Cluster_Thread | [task.check] vbucket map not ready after try 4
2019-12-10 23:14:29 | INFO | MainProcess | Cluster_Thread | [bucket_helper.wait_for_memcached] waiting for memcached bucket : default in 127.0.0.1 to accept set ops
2019-12-10 23:14:29 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 127.0.0.1:12000 default
2019-12-10 23:14:29 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 127.0.0.1:12000 default
2019-12-10 23:14:29 | WARNING | MainProcess | Cluster_Thread | [task.check] vbucket map not ready after try 5
2019-12-10 23:14:29 | INFO | MainProcess | test_thread | [basetestcase.setUp] ==============  basetestcase setup was finished for test #1 test_multi_create_query_explain_drop_index ==============
2019-12-10 23:14:29 | INFO | MainProcess | test_thread | [basetestcase.print_cluster_stats] ------- Cluster statistics -------
2019-12-10 23:14:29 | INFO | MainProcess | test_thread | [basetestcase.print_cluster_stats] 127.0.0.1:9000 => {'swap_mem_used': 0, 'cpu_utilization': 19.27083333333333, 'mem_free': 13981556736, 'swap_mem_total': 0, 'mem_total': 15466930176, 'services': [u'index', u'kv', u'n1ql']}
2019-12-10 23:14:29 | INFO | MainProcess | test_thread | [basetestcase.print_cluster_stats] --- End of cluster statistics ---
2019-12-10 23:14:29 | INFO | MainProcess | test_thread | [newtuq.setUp] Initial status of 127.0.0.1 cluster is warmup
2019-12-10 23:14:29 | INFO | MainProcess | test_thread | [newtuq.setUp] Waiting for cluster to become healthy
2019-12-10 23:14:29 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 5 secs.  ...
2019-12-10 23:14:34 | INFO | MainProcess | test_thread | [newtuq.setUp] current status of 127.0.0.1  is healthy
2019-12-10 23:14:34 | INFO | MainProcess | test_thread | [newtuq.setUp] Initial status of 127.0.0.1 cluster is healthy
2019-12-10 23:14:34 | INFO | MainProcess | test_thread | [newtuq.setUp] current status of 127.0.0.1  is healthy
2019-12-10 23:14:34 | INFO | MainProcess | test_thread | [newtuq.setUp] Initial status of 127.0.0.1 cluster is healthy
2019-12-10 23:14:34 | INFO | MainProcess | test_thread | [newtuq.setUp] current status of 127.0.0.1  is healthy
2019-12-10 23:14:34 | INFO | MainProcess | test_thread | [newtuq.setUp] Initial status of 127.0.0.1 cluster is healthy
2019-12-10 23:14:34 | INFO | MainProcess | test_thread | [newtuq.setUp] current status of 127.0.0.1  is healthy
2019-12-10 23:14:34 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of index nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
2019-12-10 23:14:34 | INFO | MainProcess | test_thread | [rest_client.set_index_settings_internal] {'indexer.settings.storage_mode': 'forestdb'} set
2019-12-10 23:14:34 | INFO | MainProcess | test_thread | [rest_client.set_index_settings_internal] {'indexer.api.enableTestServer': True} set
2019-12-10 23:14:34 | INFO | MainProcess | test_thread | [remote_util.__init__] connecting to 127.0.0.1 with username:Administrator 
2019-12-10 23:14:34 | INFO | MainProcess | test_thread | [remote_util.__init__] Connected to 127.0.0.1
2019-12-10 23:14:36 | INFO | MainProcess | test_thread | [basetestcase.load] create 2016 to default documents...
2019-12-10 23:14:36 | INFO | MainProcess | test_thread | [data_helper.direct_client] creating direct client 127.0.0.1:12000 default
2019-12-10 23:14:37 | INFO | MainProcess | test_thread | [basetestcase.load] LOAD IS FINISHED
2019-12-10 23:14:37 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
2019-12-10 23:14:37 | INFO | MainProcess | test_thread | [newtuq.setUp] ip:127.0.0.1 port:9000 ssh_username:Administrator
2019-12-10 23:14:37 | INFO | MainProcess | test_thread | [tuq_helper.run_cbq_query] RUN QUERY SELECT * FROM system:indexes where name = '#primary'
2019-12-10 23:14:37 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
2019-12-10 23:14:37 | INFO | MainProcess | test_thread | [tuq_helper.run_cbq_query] TOTAL ELAPSED TIME: 88.620343ms
2019-12-10 23:14:37 | INFO | MainProcess | test_thread | [tuq_helper.run_cbq_query] RUN QUERY CREATE PRIMARY INDEX ON default  USING GSI
2019-12-10 23:14:37 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=CREATE+PRIMARY+INDEX+ON+default++USING+GSI
2019-12-10 23:14:38 | INFO | MainProcess | test_thread | [tuq_helper.run_cbq_query] TOTAL ELAPSED TIME: 670.382251ms
2019-12-10 23:14:38 | INFO | MainProcess | test_thread | [tuq_helper.run_cbq_query] RUN QUERY SELECT * FROM system:indexes where name = '#primary'
2019-12-10 23:14:38 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
2019-12-10 23:14:38 | INFO | MainProcess | test_thread | [tuq_helper.run_cbq_query] TOTAL ELAPSED TIME: 12.89606ms
2019-12-10 23:14:38 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of index nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
2019-12-10 23:14:39 | INFO | MainProcess | Cluster_Thread | [tuq_helper.run_cbq_query] RUN QUERY CREATE INDEX employee285abd27aca94fd7be5a79627190cad6job_title ON default(job_title) USING GSI  WITH {'nodes': ['127.0.0.1:9000'], 'defer_build': True}
2019-12-10 23:14:39 | INFO | MainProcess | Cluster_Thread | [rest_client.query_tool] query params : statement=CREATE+INDEX+employee285abd27aca94fd7be5a79627190cad6job_title+ON+default%28job_title%29+USING+GSI++WITH+%7B%27nodes%27%3A+%5B%27127.0.0.1%3A9000%27%5D%2C+%27defer_build%27%3A+True%7D
2019-12-10 23:14:39 | INFO | MainProcess | Cluster_Thread | [tuq_helper.run_cbq_query] TOTAL ELAPSED TIME: 234.727738ms
2019-12-10 23:14:39 | INFO | MainProcess | test_thread | [base_2i.async_build_index] BUILD INDEX on default(employee285abd27aca94fd7be5a79627190cad6job_title) USING GSI
2019-12-10 23:14:40 | INFO | MainProcess | Cluster_Thread | [tuq_helper.run_cbq_query] RUN QUERY BUILD INDEX on default(employee285abd27aca94fd7be5a79627190cad6job_title) USING GSI
2019-12-10 23:14:40 | INFO | MainProcess | Cluster_Thread | [rest_client.query_tool] query params : statement=BUILD+INDEX+on+default%28employee285abd27aca94fd7be5a79627190cad6job_title%29+USING+GSI
2019-12-10 23:14:41 | INFO | MainProcess | Cluster_Thread | [tuq_helper.run_cbq_query] TOTAL ELAPSED TIME: 51.746947ms
2019-12-10 23:14:42 | INFO | MainProcess | Cluster_Thread | [tuq_helper.run_cbq_query] RUN QUERY SELECT * FROM system:indexes where name = 'employee285abd27aca94fd7be5a79627190cad6job_title'
2019-12-10 23:14:42 | INFO | MainProcess | Cluster_Thread | [rest_client.query_tool] query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee285abd27aca94fd7be5a79627190cad6job_title%27
2019-12-10 23:14:42 | INFO | MainProcess | Cluster_Thread | [tuq_helper.run_cbq_query] TOTAL ELAPSED TIME: 13.984068ms
2019-12-10 23:14:43 | INFO | MainProcess | Cluster_Thread | [tuq_helper.run_cbq_query] RUN QUERY SELECT * FROM system:indexes where name = 'employee285abd27aca94fd7be5a79627190cad6job_title'
2019-12-10 23:14:43 | INFO | MainProcess | Cluster_Thread | [rest_client.query_tool] query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee285abd27aca94fd7be5a79627190cad6job_title%27
2019-12-10 23:14:43 | INFO | MainProcess | Cluster_Thread | [tuq_helper.run_cbq_query] TOTAL ELAPSED TIME: 13.56672ms
2019-12-10 23:14:43 | INFO | MainProcess | Cluster_Thread | [task.execute]  <<<<< START Executing Query EXPLAIN SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
2019-12-10 23:14:43 | INFO | MainProcess | Cluster_Thread | [tuq_helper.run_cbq_query] RUN QUERY EXPLAIN SELECT * FROM default WHERE  job_title = "Sales" 
2019-12-10 23:14:43 | INFO | MainProcess | Cluster_Thread | [rest_client.query_tool] query params : statement=EXPLAIN+SELECT+%2A+FROM+default+WHERE++job_title+%3D+%22Sales%22+
2019-12-10 23:14:43 | INFO | MainProcess | Cluster_Thread | [tuq_helper.run_cbq_query] TOTAL ELAPSED TIME: 4.345209ms
2019-12-10 23:14:43 | INFO | MainProcess | Cluster_Thread | [task.execute] {u'status': u'success', u'metrics': {u'elapsedTime': u'4.345209ms', u'executionTime': u'4.254557ms', u'resultSize': 727, u'resultCount': 1}, u'results': [{u'text': u'SELECT * FROM default WHERE  job_title = "Sales"', u'plan': {u'#operator': u'Sequence', u'~children': [{u'index': u'employee285abd27aca94fd7be5a79627190cad6job_title', u'namespace': u'default', u'keyspace': u'default', u'#operator': u'IndexScan3', u'index_projection': {u'primary_key': True}, u'spans': [{u'range': [{u'high': u'"Sales"', u'low': u'"Sales"', u'inclusion': 3}], u'exact': True}], u'using': u'gsi', u'index_id': u'a7f0937dbcdac0ae'}, {u'keyspace': u'default', u'#operator': u'Fetch', u'namespace': u'default'}, {u'#operator': u'Parallel', u'~child': {u'#operator': u'Sequence', u'~children': [{u'#operator': u'Filter', u'condition': u'((`default`.`job_title`) = "Sales")'}, {u'#operator': u'InitialProject', u'result_terms': [{u'expr': u'self', u'star': True}]}, {u'#operator': u'FinalProject'}]}}]}}], u'requestID': u'bf7b2ecf-b249-49e3-b12b-bb18dd2bde9b', u'signature': u'json'}
2019-12-10 23:14:43 | INFO | MainProcess | Cluster_Thread | [task.execute]  <<<<< Done Executing Query EXPLAIN SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
2019-12-10 23:14:43 | INFO | MainProcess | Cluster_Thread | [task.check]  <<<<< Done VERIFYING Query EXPLAIN SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
2019-12-10 23:14:43 | INFO | MainProcess | test_thread | [base_2i.async_query_using_index] Query : SELECT * FROM default WHERE  job_title = "Sales" 
2019-12-10 23:14:43 | INFO | MainProcess | test_thread | [tuq_generators.generate_expected_result] FROM clause ===== is default
2019-12-10 23:14:43 | INFO | MainProcess | test_thread | [tuq_generators.generate_expected_result] WHERE clause ===== is   doc["job_title"] == "Sales" 
2019-12-10 23:14:43 | INFO | MainProcess | test_thread | [tuq_generators.generate_expected_result] UNNEST clause ===== is None
2019-12-10 23:14:43 | INFO | MainProcess | test_thread | [tuq_generators.generate_expected_result] SELECT clause ===== is {"*" : doc,}
2019-12-10 23:14:44 | INFO | MainProcess | Cluster_Thread | [task.execute]  <<<<< START Executing Query SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
2019-12-10 23:14:44 | INFO | MainProcess | Cluster_Thread | [tuq_helper.run_cbq_query] RUN QUERY SELECT * FROM default WHERE  job_title = "Sales" 
2019-12-10 23:14:44 | INFO | MainProcess | Cluster_Thread | [rest_client.query_tool] query params : scan_consistency=request_plus&statement=SELECT+%2A+FROM+default+WHERE++job_title+%3D+%22Sales%22+
2019-12-10 23:14:44 | INFO | MainProcess | Cluster_Thread | [tuq_helper.run_cbq_query] TOTAL ELAPSED TIME: 129.592199ms
2019-12-10 23:14:44 | INFO | MainProcess | Cluster_Thread | [tuq_helper._verify_results]  Analyzing Actual Result
2019-12-10 23:14:44 | INFO | MainProcess | Cluster_Thread | [tuq_helper._verify_results]  Analyzing Expected Result
2019-12-10 23:14:44 | INFO | MainProcess | Cluster_Thread | [task.execute]  <<<<< Done Executing Query SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
2019-12-10 23:14:44 | INFO | MainProcess | Cluster_Thread | [task.check]  <<<<< Done VERIFYING Query SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
2019-12-10 23:14:45 | INFO | MainProcess | Cluster_Thread | [tuq_helper.run_cbq_query] RUN QUERY SELECT * FROM system:indexes where name = 'employee285abd27aca94fd7be5a79627190cad6job_title'
2019-12-10 23:14:45 | INFO | MainProcess | Cluster_Thread | [rest_client.query_tool] query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee285abd27aca94fd7be5a79627190cad6job_title%27
2019-12-10 23:14:45 | INFO | MainProcess | Cluster_Thread | [tuq_helper.run_cbq_query] TOTAL ELAPSED TIME: 23.923228ms
2019-12-10 23:14:45 | INFO | MainProcess | Cluster_Thread | [tuq_helper.run_cbq_query] RUN QUERY DROP INDEX default.employee285abd27aca94fd7be5a79627190cad6job_title USING GSI
2019-12-10 23:14:45 | INFO | MainProcess | Cluster_Thread | [rest_client.query_tool] query params : statement=DROP+INDEX+default.employee285abd27aca94fd7be5a79627190cad6job_title+USING+GSI
2019-12-10 23:14:45 | INFO | MainProcess | Cluster_Thread | [tuq_helper.run_cbq_query] TOTAL ELAPSED TIME: 31.808774ms
2019-12-10 23:14:45 | INFO | MainProcess | Cluster_Thread | [tuq_helper.run_cbq_query] RUN QUERY SELECT * FROM system:indexes where name = 'employee285abd27aca94fd7be5a79627190cad6job_title'
2019-12-10 23:14:45 | INFO | MainProcess | Cluster_Thread | [rest_client.query_tool] query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee285abd27aca94fd7be5a79627190cad6job_title%27
2019-12-10 23:14:45 | INFO | MainProcess | Cluster_Thread | [tuq_helper.run_cbq_query] TOTAL ELAPSED TIME: 9.87525ms
2019-12-10 23:14:45 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of index nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
2019-12-10 23:14:45 | INFO | MainProcess | test_thread | [remote_util.__init__] connecting to 127.0.0.1 with username:Administrator 
2019-12-10 23:14:45 | INFO | MainProcess | test_thread | [remote_util.__init__] Connected to 127.0.0.1
2019-12-10 23:14:45 | INFO | MainProcess | test_thread | [rest_client.diag_eval] /diag/eval status on 127.0.0.1:9000: True content: "/opt/build/ns_server/logs/n_0" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))).
2019-12-10 23:14:45 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 127.0.0.1: zgrep "panic" "/opt/build/ns_server/logs/n_0"/indexer.log* | wc -l
2019-12-10 23:14:45 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully
2019-12-10 23:14:45 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of kv nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
2019-12-10 23:14:45 | INFO | MainProcess | test_thread | [remote_util.__init__] connecting to 127.0.0.1 with username:Administrator 
2019-12-10 23:14:45 | INFO | MainProcess | test_thread | [remote_util.__init__] Connected to 127.0.0.1
2019-12-10 23:14:46 | INFO | MainProcess | test_thread | [rest_client.diag_eval] /diag/eval status on 127.0.0.1:9000: True content: "/opt/build/ns_server/logs/n_0" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))).
2019-12-10 23:14:46 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 127.0.0.1: zgrep "panic" "/opt/build/ns_server/logs/n_0"/projector.log* | wc -l
2019-12-10 23:14:46 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully
2019-12-10 23:14:46 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of n1ql nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
2019-12-10 23:14:46 | INFO | MainProcess | test_thread | [tuq_helper.drop_primary_index] CHECK FOR PRIMARY INDEXES
2019-12-10 23:14:46 | INFO | MainProcess | test_thread | [tuq_helper.drop_primary_index] DROP PRIMARY INDEX ON default USING GSI
2019-12-10 23:14:46 | INFO | MainProcess | test_thread | [tuq_helper.run_cbq_query] RUN QUERY SELECT * FROM system:indexes where name = '#primary'
2019-12-10 23:14:46 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
2019-12-10 23:14:46 | INFO | MainProcess | test_thread | [tuq_helper.run_cbq_query] TOTAL ELAPSED TIME: 8.240873ms
2019-12-10 23:14:46 | INFO | MainProcess | test_thread | [tuq_helper.run_cbq_query] RUN QUERY DROP PRIMARY INDEX ON default USING GSI
2019-12-10 23:14:46 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=DROP+PRIMARY+INDEX+ON+default+USING+GSI
2019-12-10 23:14:46 | INFO | MainProcess | test_thread | [tuq_helper.run_cbq_query] TOTAL ELAPSED TIME: 35.405076ms
2019-12-10 23:14:46 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 127.0.0.1: rm -f /opt/couchbase/var/lib/couchbase/meta;rm -f /tmp/log_upr_client.sock
2019-12-10 23:14:46 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully
2019-12-10 23:14:46 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 127.0.0.1: killall cbq-engine
2019-12-10 23:14:46 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully
2019-12-10 23:14:46 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 127.0.0.1: killall tuqtng
2019-12-10 23:14:46 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully
2019-12-10 23:14:46 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 127.0.0.1: killall indexer
2019-12-10 23:14:46 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully
2019-12-10 23:14:46 | INFO | MainProcess | test_thread | [basetestcase.print_cluster_stats] ------- Cluster statistics -------
2019-12-10 23:14:46 | INFO | MainProcess | test_thread | [basetestcase.print_cluster_stats] 127.0.0.1:9000 => {'swap_mem_used': 0, 'cpu_utilization': 12.89398280802292, 'mem_free': 13832314880, 'swap_mem_total': 0, 'mem_total': 15466930176, 'services': [u'index', u'kv', u'n1ql']}
2019-12-10 23:14:46 | INFO | MainProcess | test_thread | [basetestcase.print_cluster_stats] --- End of cluster statistics ---
2019-12-10 23:14:46 | INFO | MainProcess | test_thread | [basetestcase.tearDown] ==============  basetestcase cleanup was started for test #1 test_multi_create_query_explain_drop_index ==============
2019-12-10 23:14:46 | INFO | MainProcess | test_thread | [bucket_helper.delete_all_buckets_or_assert] deleting existing buckets [u'default'] on 127.0.0.1
2019-12-10 23:14:46 | INFO | MainProcess | test_thread | [bucket_helper.delete_all_buckets_or_assert] remove bucket default ...
2019-12-10 23:14:47 | INFO | MainProcess | test_thread | [bucket_helper.delete_all_buckets_or_assert] deleted bucket : default from 127.0.0.1
2019-12-10 23:14:47 | INFO | MainProcess | test_thread | [bucket_helper.wait_for_bucket_deletion] waiting for bucket deletion to complete....
2019-12-10 23:14:47 | INFO | MainProcess | test_thread | [rest_client.bucket_exists] node 127.0.0.1 existing buckets : []
2019-12-10 23:14:47 | INFO | MainProcess | test_thread | [bucket_helper.delete_all_buckets_or_assert] sleep 2 seconds to make sure all buckets ([u'default']) were deleted completely.
2019-12-10 23:14:49 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] waiting for ns_server @ 127.0.0.1:9000
2019-12-10 23:14:49 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] ns_server @ 127.0.0.1:9000 is running
2019-12-10 23:14:49 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] waiting for ns_server @ 127.0.0.1:9001
2019-12-10 23:14:49 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] ns_server @ 127.0.0.1:9001 is running
2019-12-10 23:14:49 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] waiting for ns_server @ 127.0.0.1:9002
2019-12-10 23:14:49 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] ns_server @ 127.0.0.1:9002 is running
2019-12-10 23:14:49 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] waiting for ns_server @ 127.0.0.1:9003
2019-12-10 23:14:49 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] ns_server @ 127.0.0.1:9003 is running
2019-12-10 23:14:49 | INFO | MainProcess | test_thread | [basetestcase.tearDown] ==============  basetestcase cleanup was finished for test #1 test_multi_create_query_explain_drop_index ==============
ok

----------------------------------------------------------------------
Ran 1 test in 46.544s

OK
test_multi_create_query_explain_drop_index (2i.indexscans_2i.SecondaryIndexingScanTests) ... Cluster instance shutdown with force
summary so far suite 2i.indexscans_2i.SecondaryIndexingScanTests , pass 1 , fail 0
testrunner logs, diags and results are available under /opt/build/testrunner/logs/testrunner-19-Dec-10_23-14-02/test_1
Logs will be stored at /opt/build/testrunner/logs/testrunner-19-Dec-10_23-14-02/test_2

./testrunner -i b/resources/dev-4-nodes-xdcr_n1ql_2i.ini -p makefile=True -t 2i.indexscans_2i.SecondaryIndexingScanTests.test_multi_create_query_explain_drop_index,groups=simple:equals:no_orderby_groupby:range,dataset=default,doc-per-day=1,use_gsi_for_primary=True,use_gsi_for_secondary=False,GROUP=views

Test Input params:
{'doc-per-day': '1', 'use_gsi_for_primary': 'True', 'GROUP': 'views', 'use_gsi_for_secondary': 'False', 'gsi_type': 'forestdb', 'makefile': 'True', 'conf_file': 'conf/simple_2i_n1ql.conf', 'dataset': 'default', 'cluster_name': 'dev-4-nodes-xdcr_n1ql_2i', 'ini': 'b/resources/dev-4-nodes-xdcr_n1ql_2i.ini', 'groups': 'simple:equals:no_orderby_groupby:range', 'case_number': 2, 'num_nodes': 4, 'logs_folder': '/opt/build/testrunner/logs/testrunner-19-Dec-10_23-14-02/test_2', 'spec': 'simple_2i_n1ql'}
[2019-12-10 23:14:49,801] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:14:49,801] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:14:50,046] - [rest_client:2127] INFO - Node version in cluster 6.5.0-4928-rel-enterprise
[2019-12-10 23:14:50,062] - [rest_client:2137] INFO - Node versions in cluster [u'6.5.0-4928-rel-enterprise']
[2019-12-10 23:14:50,062] - [basetestcase:218] INFO - ==============  basetestcase setup was started for test #2 test_multi_create_query_explain_drop_index==============
[2019-12-10 23:14:50,161] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9000
[2019-12-10 23:14:50,178] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9000 is running
[2019-12-10 23:14:50,185] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9001
[2019-12-10 23:14:50,194] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9001 is running
[2019-12-10 23:14:50,201] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9002
[2019-12-10 23:14:50,207] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9002 is running
[2019-12-10 23:14:50,213] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9003
[2019-12-10 23:14:50,220] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9003 is running
[2019-12-10 23:14:50,221] - [basetestcase:265] INFO - initializing cluster
[2019-12-10 23:14:50,823] - [task:132] INFO - server: ip:127.0.0.1 port:9000 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_0@cb.local', 'uptime': u'44', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9000', 'storage': [], 'moxi': 11211, 'port': u'9000', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12000, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'index', u'kv', u'n1ql'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13866553344, 'memoryTotal': 15466930176, 'memoryQuota': 7650, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:14:50,829] - [task:171] INFO - quota for index service will be 256 MB
[2019-12-10 23:14:50,829] - [task:173] INFO - set index quota to node 127.0.0.1 
[2019-12-10 23:14:50,829] - [rest_client:965] INFO - pools/default params : indexMemoryQuota=256
[2019-12-10 23:14:50,842] - [rest_client:953] INFO - pools/default params : memoryQuota=7650
[2019-12-10 23:14:50,850] - [rest_client:931] INFO - /node/controller/setupServices params on 127.0.0.1:9000:services=kv%2Cindex%2Cn1ql&password=asdasd&user=Administrator&hostname=127.0.0.1%3A9000
[2019-12-10 23:14:50,867] - [rest_client:840] ERROR - POST http://127.0.0.1:9000//node/controller/setupServices body: services=kv%2Cindex%2Cn1ql&password=asdasd&user=Administrator&hostname=127.0.0.1%3A9000 headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown ["cannot change node services after cluster is provisioned"] auth: Administrator:asdasd
[2019-12-10 23:14:50,870] - [rest_client:936] INFO - This node is already provisioned with services, we do not consider this as failure for test case
[2019-12-10 23:14:50,870] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9000:username=Administrator&password=asdasd&port=9000
[2019-12-10 23:14:50,932] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:14:50,933] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:14:51,172] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9000/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:14:51,219] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:14:51,227] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9000: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:14:51,227] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:14:51,233] - [rest_client:840] ERROR - POST http://127.0.0.1:9000/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:14:51,261] - [task:132] INFO - server: ip:127.0.0.1 port:9001 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_1@cb.local', 'uptime': u'44', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9001', 'storage': [], 'moxi': 11211, 'port': u'9001', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12002, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13866811392, 'memoryTotal': 15466930176, 'memoryQuota': 7906, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:14:51,270] - [rest_client:953] INFO - pools/default params : memoryQuota=7906
[2019-12-10 23:14:51,278] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9001:username=Administrator&password=asdasd&port=9001
[2019-12-10 23:14:51,338] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:14:51,338] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:14:51,582] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9001/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:14:51,628] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:14:51,636] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9001: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:14:51,636] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:14:51,642] - [rest_client:840] ERROR - POST http://127.0.0.1:9001/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:14:51,668] - [task:132] INFO - server: ip:127.0.0.1 port:9002 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_2@cb.local', 'uptime': u'44', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9002', 'storage': [], 'moxi': 11211, 'port': u'9002', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12004, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13864140800, 'memoryTotal': 15466930176, 'memoryQuota': 7906, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:14:51,678] - [rest_client:953] INFO - pools/default params : memoryQuota=7906
[2019-12-10 23:14:51,686] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9002:username=Administrator&password=asdasd&port=9002
[2019-12-10 23:14:51,744] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:14:51,744] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:14:51,984] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9002/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:14:52,029] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:14:52,036] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9002: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:14:52,037] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:14:52,042] - [rest_client:840] ERROR - POST http://127.0.0.1:9002/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:14:52,069] - [task:132] INFO - server: ip:127.0.0.1 port:9003 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_3@cb.local', 'uptime': u'44', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9003', 'storage': [], 'moxi': 11211, 'port': u'9003', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12006, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13863505920, 'memoryTotal': 15466930176, 'memoryQuota': 7906, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:14:52,078] - [rest_client:953] INFO - pools/default params : memoryQuota=7906
[2019-12-10 23:14:52,087] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9003:username=Administrator&password=asdasd&port=9003
[2019-12-10 23:14:52,146] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:14:52,146] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:14:52,376] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9003/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:14:52,421] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:14:52,432] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9003: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:14:52,433] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:14:52,438] - [rest_client:840] ERROR - POST http://127.0.0.1:9003/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:14:52,472] - [basetestcase:2218] INFO - **** add built-in 'cbadminbucket' user to node 127.0.0.1 ****
[2019-12-10 23:14:52,573] - [basetestcase:530] INFO - sleep for 5 secs.  ...
[2019-12-10 23:14:57,578] - [basetestcase:2223] INFO - **** add 'admin' role to 'cbadminbucket' user ****
[2019-12-10 23:14:57,602] - [basetestcase:291] INFO - done initializing cluster
[2019-12-10 23:14:57,606] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:14:57,607] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:14:57,838] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9000/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:14:57,885] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:14:57,886] - [basetestcase:2770] INFO - Enabled diag/eval for non-local hosts from 127.0.0.1
[2019-12-10 23:14:58,491] - [rest_client:2430] INFO - http://127.0.0.1:9000/pools/default/buckets with param: bucketType=membase&evictionPolicy=valueOnly&threadsNumber=3&ramQuotaMB=7650&authType=sasl&name=default&flushEnabled=1&replicaNumber=1&compressionMode=passive&saslPassword=None&replicaIndex=1
[2019-12-10 23:14:58,505] - [rest_client:2455] INFO - 0.01 seconds to create bucket default
[2019-12-10 23:14:58,505] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:14:58,646] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:14:58,726] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:14:58,772] - [task:351] WARNING - vbucket map not ready after try 0
[2019-12-10 23:14:58,772] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:14:58,889] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:14:58,975] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:14:59,017] - [task:351] WARNING - vbucket map not ready after try 1
[2019-12-10 23:14:59,017] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:14:59,105] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:14:59,181] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:14:59,223] - [task:351] WARNING - vbucket map not ready after try 2
[2019-12-10 23:14:59,223] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:14:59,313] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:14:59,400] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:14:59,441] - [task:351] WARNING - vbucket map not ready after try 3
[2019-12-10 23:14:59,441] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:14:59,533] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:14:59,626] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:14:59,688] - [task:351] WARNING - vbucket map not ready after try 4
[2019-12-10 23:14:59,688] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:14:59,785] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:14:59,875] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:14:59,919] - [task:351] WARNING - vbucket map not ready after try 5
[2019-12-10 23:14:59,944] - [basetestcase:383] INFO - ==============  basetestcase setup was finished for test #2 test_multi_create_query_explain_drop_index ==============
[2019-12-10 23:14:59,968] - [basetestcase:397] INFO - ------- Cluster statistics -------
[2019-12-10 23:14:59,969] - [basetestcase:399] INFO - 127.0.0.1:9000 => {'swap_mem_used': 0, 'cpu_utilization': 11.34020618556701, 'mem_free': 13907304448, 'swap_mem_total': 0, 'mem_total': 15466930176, 'services': [u'index', u'kv', u'n1ql']}
[2019-12-10 23:14:59,969] - [basetestcase:400] INFO - --- End of cluster statistics ---
[2019-12-10 23:14:59,980] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is warmup
[2019-12-10 23:14:59,981] - [newtuq:22] INFO - Waiting for cluster to become healthy
[2019-12-10 23:14:59,981] - [basetestcase:530] INFO - sleep for 5 secs.  ...
[2019-12-10 23:15:04,992] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:15:05,005] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is healthy
[2019-12-10 23:15:05,006] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:15:05,019] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is healthy
[2019-12-10 23:15:05,019] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:15:05,033] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is healthy
[2019-12-10 23:15:05,033] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:15:05,045] - [basetestcase:2436] INFO - list of index nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:15:05,068] - [rest_client:1706] INFO - {'indexer.settings.storage_mode': 'forestdb'} set
[2019-12-10 23:15:05,104] - [rest_client:1706] INFO - {'indexer.api.enableTestServer': True} set
[2019-12-10 23:15:05,107] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:15:05,108] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:15:06,465] - [basetestcase:2545] INFO - create 2016 to default documents...
[2019-12-10 23:15:06,520] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:15:08,164] - [basetestcase:2557] INFO - LOAD IS FINISHED
[2019-12-10 23:15:08,202] - [basetestcase:2436] INFO - list of n1ql nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:15:08,202] - [newtuq:83] INFO - ip:127.0.0.1 port:9000 ssh_username:Administrator
[2019-12-10 23:15:08,202] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2019-12-10 23:15:08,210] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2019-12-10 23:15:08,301] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 87.343129ms
[2019-12-10 23:15:08,301] - [tuq_helper:147] INFO - RUN QUERY CREATE PRIMARY INDEX ON default  USING GSI
[2019-12-10 23:15:08,308] - [rest_client:3465] INFO - query params : statement=CREATE+PRIMARY+INDEX+ON+default++USING+GSI
[2019-12-10 23:15:09,088] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 776.456691ms
[2019-12-10 23:15:09,088] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2019-12-10 23:15:09,099] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2019-12-10 23:15:09,114] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 11.175264ms
[2019-12-10 23:15:09,145] - [basetestcase:2436] INFO - list of index nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:15:10,132] - [tuq_helper:147] INFO - RUN QUERY CREATE INDEX employee25dc74033aac43e9ab35f53a53deeaf0job_title ON default(job_title) USING VIEW 
[2019-12-10 23:15:10,140] - [rest_client:3465] INFO - query params : statement=CREATE+INDEX+employee25dc74033aac43e9ab35f53a53deeaf0job_title+ON+default%28job_title%29+USING+VIEW+
[2019-12-10 23:15:10,739] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 596.144846ms
[2019-12-10 23:15:10,739] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee25dc74033aac43e9ab35f53a53deeaf0job_title'
[2019-12-10 23:15:10,747] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee25dc74033aac43e9ab35f53a53deeaf0job_title%27
[2019-12-10 23:15:10,802] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 52.827619ms
[2019-12-10 23:15:11,804] - [task:2607] INFO -  <<<<< START Executing Query EXPLAIN SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2019-12-10 23:15:11,804] - [tuq_helper:147] INFO - RUN QUERY EXPLAIN SELECT * FROM default WHERE  job_title = "Sales" 
[2019-12-10 23:15:11,812] - [rest_client:3465] INFO - query params : statement=EXPLAIN+SELECT+%2A+FROM+default+WHERE++job_title+%3D+%22Sales%22+
[2019-12-10 23:15:11,817] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 2.833785ms
[2019-12-10 23:15:11,818] - [task:2616] INFO - {u'status': u'success', u'metrics': {u'elapsedTime': u'2.833785ms', u'executionTime': u'2.716211ms', u'resultSize': 722, u'resultCount': 1}, u'results': [{u'text': u'SELECT * FROM default WHERE  job_title = "Sales"', u'plan': {u'#operator': u'Sequence', u'~children': [{u'index': u'employee25dc74033aac43e9ab35f53a53deeaf0job_title', u'namespace': u'default', u'keyspace': u'default', u'#operator': u'IndexScan', u'spans': [{u'Range': {u'High': [u'"Sales"'], u'Low': [u'"Sales"'], u'Inclusion': 3}, u'Exact': True}], u'using': u'view', u'index_id': u'employee25dc74033aac43e9ab35f53a53deeaf0job_title'}, {u'keyspace': u'default', u'#operator': u'Fetch', u'namespace': u'default'}, {u'#operator': u'Parallel', u'~child': {u'#operator': u'Sequence', u'~children': [{u'#operator': u'Filter', u'condition': u'((`default`.`job_title`) = "Sales")'}, {u'#operator': u'InitialProject', u'result_terms': [{u'expr': u'self', u'star': True}]}, {u'#operator': u'FinalProject'}]}}]}}], u'requestID': u'355fe797-8794-49e8-8723-f19e84e7746b', u'signature': u'json'}
[2019-12-10 23:15:11,818] - [task:2617] INFO -  <<<<< Done Executing Query EXPLAIN SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2019-12-10 23:15:11,818] - [task:2645] INFO -  <<<<< Done VERIFYING Query EXPLAIN SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2019-12-10 23:15:11,819] - [base_2i:314] INFO - Query : SELECT * FROM default WHERE  job_title = "Sales" 
[2019-12-10 23:15:11,820] - [tuq_generators:70] INFO - FROM clause ===== is default
[2019-12-10 23:15:11,820] - [tuq_generators:72] INFO - WHERE clause ===== is   doc["job_title"] == "Sales" 
[2019-12-10 23:15:11,820] - [tuq_generators:74] INFO - UNNEST clause ===== is None
[2019-12-10 23:15:11,821] - [tuq_generators:76] INFO - SELECT clause ===== is {"*" : doc,}
[2019-12-10 23:15:12,820] - [task:2607] INFO -  <<<<< START Executing Query SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2019-12-10 23:15:12,820] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM default WHERE  job_title = "Sales" 
[2019-12-10 23:15:12,827] - [rest_client:3465] INFO - query params : scan_consistency=request_plus&statement=SELECT+%2A+FROM+default+WHERE++job_title+%3D+%22Sales%22+
[2019-12-10 23:15:12,978] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 133.076908ms
[2019-12-10 23:15:12,998] - [tuq_helper:214] INFO -  Analyzing Actual Result
[2019-12-10 23:15:12,998] - [tuq_helper:216] INFO -  Analyzing Expected Result
[2019-12-10 23:15:13,001] - [task:2617] INFO -  <<<<< Done Executing Query SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2019-12-10 23:15:13,002] - [task:2645] INFO -  <<<<< Done VERIFYING Query SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2019-12-10 23:15:14,002] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee25dc74033aac43e9ab35f53a53deeaf0job_title'
[2019-12-10 23:15:14,010] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee25dc74033aac43e9ab35f53a53deeaf0job_title%27
[2019-12-10 23:15:14,029] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 16.616869ms
[2019-12-10 23:15:14,030] - [tuq_helper:147] INFO - RUN QUERY DROP INDEX default.employee25dc74033aac43e9ab35f53a53deeaf0job_title USING VIEW 
[2019-12-10 23:15:14,037] - [rest_client:3465] INFO - query params : statement=DROP+INDEX+default.employee25dc74033aac43e9ab35f53a53deeaf0job_title+USING+VIEW+
[2019-12-10 23:15:14,053] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 13.297237ms
[2019-12-10 23:15:14,053] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee25dc74033aac43e9ab35f53a53deeaf0job_title'
[2019-12-10 23:15:14,062] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee25dc74033aac43e9ab35f53a53deeaf0job_title%27
[2019-12-10 23:15:14,074] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 8.756614ms
[2019-12-10 23:15:14,100] - [basetestcase:2436] INFO - list of index nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:15:14,104] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:15:14,104] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:15:14,414] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9000: True content: "/opt/build/ns_server/logs/n_0" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))).
[2019-12-10 23:15:14,415] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: zgrep "panic" "/opt/build/ns_server/logs/n_0"/indexer.log* | wc -l
[2019-12-10 23:15:14,465] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:15:14,480] - [basetestcase:2436] INFO - list of kv nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:15:14,484] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:15:14,484] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:15:14,820] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9000: True content: "/opt/build/ns_server/logs/n_0" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))).
[2019-12-10 23:15:14,821] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: zgrep "panic" "/opt/build/ns_server/logs/n_0"/projector.log* | wc -l
[2019-12-10 23:15:14,878] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:15:14,895] - [basetestcase:2436] INFO - list of n1ql nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:15:14,895] - [tuq_helper:516] INFO - CHECK FOR PRIMARY INDEXES
[2019-12-10 23:15:14,895] - [tuq_helper:523] INFO - DROP PRIMARY INDEX ON default USING GSI
[2019-12-10 23:15:14,896] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2019-12-10 23:15:14,904] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2019-12-10 23:15:14,917] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 10.8047ms
[2019-12-10 23:15:14,918] - [tuq_helper:147] INFO - RUN QUERY DROP PRIMARY INDEX ON default USING GSI
[2019-12-10 23:15:14,925] - [rest_client:3465] INFO - query params : statement=DROP+PRIMARY+INDEX+ON+default+USING+GSI
[2019-12-10 23:15:14,963] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 33.222093ms
[2019-12-10 23:15:14,964] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: rm -f /opt/couchbase/var/lib/couchbase/meta;rm -f /tmp/log_upr_client.sock
[2019-12-10 23:15:15,047] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:15:15,049] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: killall cbq-engine
[2019-12-10 23:15:15,100] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:15:15,101] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: killall tuqtng
[2019-12-10 23:15:15,170] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:15:15,172] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: killall indexer
[2019-12-10 23:15:15,247] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:15:15,303] - [basetestcase:397] INFO - ------- Cluster statistics -------
[2019-12-10 23:15:15,317] - [basetestcase:399] INFO - 127.0.0.1:9000 => {'swap_mem_used': 0, 'cpu_utilization': 18.33333333333333, 'mem_free': 13743632384, 'swap_mem_total': 0, 'mem_total': 15466930176, 'services': [u'index', u'kv', u'n1ql']}
[2019-12-10 23:15:15,317] - [basetestcase:400] INFO - --- End of cluster statistics ---
[2019-12-10 23:15:15,317] - [basetestcase:462] INFO - ==============  basetestcase cleanup was started for test #2 test_multi_create_query_explain_drop_index ==============
[2019-12-10 23:15:15,381] - [bucket_helper:143] INFO - deleting existing buckets [u'default'] on 127.0.0.1
[2019-12-10 23:15:15,381] - [bucket_helper:145] INFO - remove bucket default ...
[2019-12-10 23:15:15,877] - [bucket_helper:159] INFO - deleted bucket : default from 127.0.0.1
[2019-12-10 23:15:15,877] - [bucket_helper:237] INFO - waiting for bucket deletion to complete....
[2019-12-10 23:15:15,882] - [rest_client:133] INFO - node 127.0.0.1 existing buckets : []
[2019-12-10 23:15:15,883] - [bucket_helper:169] INFO - sleep 2 seconds to make sure all buckets ([u'default']) were deleted completely.
[2019-12-10 23:15:17,952] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9000
[2019-12-10 23:15:17,959] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9000 is running
[2019-12-10 23:15:17,967] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9001
[2019-12-10 23:15:17,973] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9001 is running
[2019-12-10 23:15:17,979] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9002
[2019-12-10 23:15:17,986] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9002 is running
[2019-12-10 23:15:17,992] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9003
[2019-12-10 23:15:17,998] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9003 is running
[2019-12-10 23:15:17,999] - [basetestcase:480] INFO - ==============  basetestcase cleanup was finished for test #2 test_multi_create_query_explain_drop_index ==============
ok

----------------------------------------------------------------------
Ran 1 test in 28.214s

OK
test_multi_create_query_explain_drop_index (2i.indexscans_2i.SecondaryIndexingScanTests) ... Cluster instance shutdown with force
summary so far suite 2i.indexscans_2i.SecondaryIndexingScanTests , pass 2 , fail 0
testrunner logs, diags and results are available under /opt/build/testrunner/logs/testrunner-19-Dec-10_23-14-02/test_2
Logs will be stored at /opt/build/testrunner/logs/testrunner-19-Dec-10_23-14-02/test_3

./testrunner -i b/resources/dev-4-nodes-xdcr_n1ql_2i.ini -p makefile=True -t 2i.indexscans_2i.SecondaryIndexingScanTests.test_multi_create_query_explain_drop_index,groups=simple:equals:no_orderby_groupby:range,dataset=default,doc-per-day=1,use_gsi_for_primary=False,use_gsi_for_secondary=True,GROUP=views

Test Input params:
{'doc-per-day': '1', 'use_gsi_for_primary': 'False', 'GROUP': 'views', 'use_gsi_for_secondary': 'True', 'gsi_type': 'forestdb', 'makefile': 'True', 'conf_file': 'conf/simple_2i_n1ql.conf', 'dataset': 'default', 'cluster_name': 'dev-4-nodes-xdcr_n1ql_2i', 'ini': 'b/resources/dev-4-nodes-xdcr_n1ql_2i.ini', 'groups': 'simple:equals:no_orderby_groupby:range', 'case_number': 3, 'num_nodes': 4, 'logs_folder': '/opt/build/testrunner/logs/testrunner-19-Dec-10_23-14-02/test_3', 'spec': 'simple_2i_n1ql'}
[2019-12-10 23:15:18,040] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:15:18,041] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:15:18,336] - [rest_client:2127] INFO - Node version in cluster 6.5.0-4928-rel-enterprise
[2019-12-10 23:15:18,351] - [rest_client:2137] INFO - Node versions in cluster [u'6.5.0-4928-rel-enterprise']
[2019-12-10 23:15:18,351] - [basetestcase:218] INFO - ==============  basetestcase setup was started for test #3 test_multi_create_query_explain_drop_index==============
[2019-12-10 23:15:18,443] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9000
[2019-12-10 23:15:18,450] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9000 is running
[2019-12-10 23:15:18,457] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9001
[2019-12-10 23:15:18,463] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9001 is running
[2019-12-10 23:15:18,470] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9002
[2019-12-10 23:15:18,476] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9002 is running
[2019-12-10 23:15:18,483] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9003
[2019-12-10 23:15:18,491] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9003 is running
[2019-12-10 23:15:18,492] - [basetestcase:265] INFO - initializing cluster
[2019-12-10 23:15:19,052] - [task:132] INFO - server: ip:127.0.0.1 port:9000 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_0@cb.local', 'uptime': u'70', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9000', 'storage': [], 'moxi': 11211, 'port': u'9000', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12000, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'index', u'kv', u'n1ql'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13776130048, 'memoryTotal': 15466930176, 'memoryQuota': 7650, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:15:19,058] - [task:171] INFO - quota for index service will be 256 MB
[2019-12-10 23:15:19,059] - [task:173] INFO - set index quota to node 127.0.0.1 
[2019-12-10 23:15:19,059] - [rest_client:965] INFO - pools/default params : indexMemoryQuota=256
[2019-12-10 23:15:19,070] - [rest_client:953] INFO - pools/default params : memoryQuota=7650
[2019-12-10 23:15:19,095] - [rest_client:931] INFO - /node/controller/setupServices params on 127.0.0.1:9000:services=kv%2Cindex%2Cn1ql&password=asdasd&user=Administrator&hostname=127.0.0.1%3A9000
[2019-12-10 23:15:19,102] - [rest_client:840] ERROR - POST http://127.0.0.1:9000//node/controller/setupServices body: services=kv%2Cindex%2Cn1ql&password=asdasd&user=Administrator&hostname=127.0.0.1%3A9000 headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown ["cannot change node services after cluster is provisioned"] auth: Administrator:asdasd
[2019-12-10 23:15:19,103] - [rest_client:936] INFO - This node is already provisioned with services, we do not consider this as failure for test case
[2019-12-10 23:15:19,103] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9000:username=Administrator&password=asdasd&port=9000
[2019-12-10 23:15:19,160] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:15:19,160] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:15:19,460] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9000/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:15:19,518] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:15:19,527] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9000: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:15:19,528] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:15:19,534] - [rest_client:840] ERROR - POST http://127.0.0.1:9000/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:15:19,561] - [task:132] INFO - server: ip:127.0.0.1 port:9001 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_1@cb.local', 'uptime': u'69', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9001', 'storage': [], 'moxi': 11211, 'port': u'9001', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12002, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13743632384, 'memoryTotal': 15466930176, 'memoryQuota': 7906, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:15:19,571] - [rest_client:953] INFO - pools/default params : memoryQuota=7906
[2019-12-10 23:15:19,580] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9001:username=Administrator&password=asdasd&port=9001
[2019-12-10 23:15:19,639] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:15:19,639] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:15:19,963] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9001/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:15:20,021] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:15:20,029] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9001: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:15:20,029] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:15:20,035] - [rest_client:840] ERROR - POST http://127.0.0.1:9001/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:15:20,064] - [task:132] INFO - server: ip:127.0.0.1 port:9002 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_2@cb.local', 'uptime': u'69', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9002', 'storage': [], 'moxi': 11211, 'port': u'9002', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12004, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13743386624, 'memoryTotal': 15466930176, 'memoryQuota': 7906, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:15:20,074] - [rest_client:953] INFO - pools/default params : memoryQuota=7906
[2019-12-10 23:15:20,082] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9002:username=Administrator&password=asdasd&port=9002
[2019-12-10 23:15:20,143] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:15:20,143] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:15:20,440] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9002/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:15:20,498] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:15:20,506] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9002: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:15:20,507] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:15:20,512] - [rest_client:840] ERROR - POST http://127.0.0.1:9002/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:15:20,539] - [task:132] INFO - server: ip:127.0.0.1 port:9003 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_3@cb.local', 'uptime': u'74', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9003', 'storage': [], 'moxi': 11211, 'port': u'9003', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12006, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13801578496, 'memoryTotal': 15466930176, 'memoryQuota': 7906, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:15:20,549] - [rest_client:953] INFO - pools/default params : memoryQuota=7906
[2019-12-10 23:15:20,557] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9003:username=Administrator&password=asdasd&port=9003
[2019-12-10 23:15:20,617] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:15:20,617] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:15:20,912] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9003/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:15:20,970] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:15:20,978] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9003: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:15:20,978] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:15:20,984] - [rest_client:840] ERROR - POST http://127.0.0.1:9003/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:15:21,013] - [basetestcase:2218] INFO - **** add built-in 'cbadminbucket' user to node 127.0.0.1 ****
[2019-12-10 23:15:21,142] - [basetestcase:530] INFO - sleep for 5 secs.  ...
[2019-12-10 23:15:26,147] - [basetestcase:2223] INFO - **** add 'admin' role to 'cbadminbucket' user ****
[2019-12-10 23:15:26,170] - [basetestcase:291] INFO - done initializing cluster
[2019-12-10 23:15:26,174] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:15:26,174] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:15:26,465] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9000/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:15:26,523] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:15:26,524] - [basetestcase:2770] INFO - Enabled diag/eval for non-local hosts from 127.0.0.1
[2019-12-10 23:15:27,037] - [rest_client:2430] INFO - http://127.0.0.1:9000/pools/default/buckets with param: bucketType=membase&evictionPolicy=valueOnly&threadsNumber=3&ramQuotaMB=7650&authType=sasl&name=default&flushEnabled=1&replicaNumber=1&compressionMode=passive&saslPassword=None&replicaIndex=1
[2019-12-10 23:15:27,054] - [rest_client:2455] INFO - 0.02 seconds to create bucket default
[2019-12-10 23:15:27,055] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:15:27,231] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:15:27,318] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:15:27,371] - [task:351] WARNING - vbucket map not ready after try 0
[2019-12-10 23:15:27,372] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:15:27,470] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:15:27,557] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:15:27,600] - [task:351] WARNING - vbucket map not ready after try 1
[2019-12-10 23:15:27,600] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:15:27,694] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:15:27,773] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:15:27,825] - [task:351] WARNING - vbucket map not ready after try 2
[2019-12-10 23:15:27,826] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:15:27,930] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:15:28,007] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:15:28,049] - [task:351] WARNING - vbucket map not ready after try 3
[2019-12-10 23:15:28,049] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:15:28,153] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:15:28,249] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:15:28,294] - [task:351] WARNING - vbucket map not ready after try 4
[2019-12-10 23:15:28,295] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:15:28,389] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:15:28,465] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:15:28,509] - [task:351] WARNING - vbucket map not ready after try 5
[2019-12-10 23:15:28,518] - [basetestcase:383] INFO - ==============  basetestcase setup was finished for test #3 test_multi_create_query_explain_drop_index ==============
[2019-12-10 23:15:28,541] - [basetestcase:397] INFO - ------- Cluster statistics -------
[2019-12-10 23:15:28,541] - [basetestcase:399] INFO - 127.0.0.1:9000 => {'swap_mem_used': 0, 'cpu_utilization': 26.5625, 'mem_free': 13807951872, 'swap_mem_total': 0, 'mem_total': 15466930176, 'services': [u'index', u'kv', u'n1ql']}
[2019-12-10 23:15:28,542] - [basetestcase:400] INFO - --- End of cluster statistics ---
[2019-12-10 23:15:28,553] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is warmup
[2019-12-10 23:15:28,554] - [newtuq:22] INFO - Waiting for cluster to become healthy
[2019-12-10 23:15:28,554] - [basetestcase:530] INFO - sleep for 5 secs.  ...
[2019-12-10 23:15:33,568] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:15:33,581] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is healthy
[2019-12-10 23:15:33,582] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:15:33,594] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is healthy
[2019-12-10 23:15:33,595] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:15:33,608] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is healthy
[2019-12-10 23:15:33,609] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:15:33,620] - [basetestcase:2436] INFO - list of index nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:15:33,643] - [rest_client:1706] INFO - {'indexer.settings.storage_mode': 'forestdb'} set
[2019-12-10 23:15:33,675] - [rest_client:1706] INFO - {'indexer.api.enableTestServer': True} set
[2019-12-10 23:15:33,679] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:15:33,679] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:15:35,154] - [basetestcase:2545] INFO - create 2016 to default documents...
[2019-12-10 23:15:35,206] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:15:36,770] - [basetestcase:2557] INFO - LOAD IS FINISHED
[2019-12-10 23:15:36,809] - [basetestcase:2436] INFO - list of n1ql nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:15:36,809] - [newtuq:83] INFO - ip:127.0.0.1 port:9000 ssh_username:Administrator
[2019-12-10 23:15:36,809] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2019-12-10 23:15:36,816] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2019-12-10 23:15:36,910] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 90.507652ms
[2019-12-10 23:15:36,910] - [tuq_helper:147] INFO - RUN QUERY CREATE PRIMARY INDEX ON default  USING VIEW 
[2019-12-10 23:15:36,919] - [rest_client:3465] INFO - query params : statement=CREATE+PRIMARY+INDEX+ON+default++USING+VIEW+
[2019-12-10 23:15:37,419] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 496.580094ms
[2019-12-10 23:15:37,419] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2019-12-10 23:15:37,426] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2019-12-10 23:15:37,441] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 12.670658ms
[2019-12-10 23:15:37,467] - [basetestcase:2436] INFO - list of index nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:15:38,456] - [tuq_helper:147] INFO - RUN QUERY CREATE INDEX employeee5aa6b473c8443a4882e99c6bf403510job_title ON default(job_title) USING GSI  WITH {'nodes': ['127.0.0.1:9000'], 'defer_build': True}
[2019-12-10 23:15:38,463] - [rest_client:3465] INFO - query params : statement=CREATE+INDEX+employeee5aa6b473c8443a4882e99c6bf403510job_title+ON+default%28job_title%29+USING+GSI++WITH+%7B%27nodes%27%3A+%5B%27127.0.0.1%3A9000%27%5D%2C+%27defer_build%27%3A+True%7D
[2019-12-10 23:15:38,662] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 195.963097ms
[2019-12-10 23:15:38,663] - [base_2i:109] INFO - BUILD INDEX on default(employeee5aa6b473c8443a4882e99c6bf403510job_title) USING GSI
[2019-12-10 23:15:39,663] - [tuq_helper:147] INFO - RUN QUERY BUILD INDEX on default(employeee5aa6b473c8443a4882e99c6bf403510job_title) USING GSI
[2019-12-10 23:15:39,671] - [rest_client:3465] INFO - query params : statement=BUILD+INDEX+on+default%28employeee5aa6b473c8443a4882e99c6bf403510job_title%29+USING+GSI
[2019-12-10 23:15:39,758] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 83.577567ms
[2019-12-10 23:15:40,759] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employeee5aa6b473c8443a4882e99c6bf403510job_title'
[2019-12-10 23:15:40,766] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employeee5aa6b473c8443a4882e99c6bf403510job_title%27
[2019-12-10 23:15:40,782] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 12.820106ms
[2019-12-10 23:15:41,783] - [task:2607] INFO -  <<<<< START Executing Query EXPLAIN SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2019-12-10 23:15:41,784] - [tuq_helper:147] INFO - RUN QUERY EXPLAIN SELECT * FROM default WHERE  job_title = "Sales" 
[2019-12-10 23:15:41,791] - [rest_client:3465] INFO - query params : statement=EXPLAIN+SELECT+%2A+FROM+default+WHERE++job_title+%3D+%22Sales%22+
[2019-12-10 23:15:41,797] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 2.922113ms
[2019-12-10 23:15:41,797] - [task:2616] INFO - {u'status': u'success', u'metrics': {u'elapsedTime': u'2.922113ms', u'executionTime': u'2.784668ms', u'resultSize': 727, u'resultCount': 1}, u'results': [{u'text': u'SELECT * FROM default WHERE  job_title = "Sales"', u'plan': {u'#operator': u'Sequence', u'~children': [{u'index': u'employeee5aa6b473c8443a4882e99c6bf403510job_title', u'namespace': u'default', u'keyspace': u'default', u'#operator': u'IndexScan3', u'index_projection': {u'primary_key': True}, u'spans': [{u'range': [{u'high': u'"Sales"', u'low': u'"Sales"', u'inclusion': 3}], u'exact': True}], u'using': u'gsi', u'index_id': u'fe3f5ea7ec55047f'}, {u'keyspace': u'default', u'#operator': u'Fetch', u'namespace': u'default'}, {u'#operator': u'Parallel', u'~child': {u'#operator': u'Sequence', u'~children': [{u'#operator': u'Filter', u'condition': u'((`default`.`job_title`) = "Sales")'}, {u'#operator': u'InitialProject', u'result_terms': [{u'expr': u'self', u'star': True}]}, {u'#operator': u'FinalProject'}]}}]}}], u'requestID': u'00213baf-5628-4a9d-8330-19d28e07a8a0', u'signature': u'json'}
[2019-12-10 23:15:41,797] - [task:2617] INFO -  <<<<< Done Executing Query EXPLAIN SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2019-12-10 23:15:41,798] - [task:2645] INFO -  <<<<< Done VERIFYING Query EXPLAIN SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2019-12-10 23:15:41,798] - [base_2i:314] INFO - Query : SELECT * FROM default WHERE  job_title = "Sales" 
[2019-12-10 23:15:41,798] - [tuq_generators:70] INFO - FROM clause ===== is default
[2019-12-10 23:15:41,799] - [tuq_generators:72] INFO - WHERE clause ===== is   doc["job_title"] == "Sales" 
[2019-12-10 23:15:41,799] - [tuq_generators:74] INFO - UNNEST clause ===== is None
[2019-12-10 23:15:41,799] - [tuq_generators:76] INFO - SELECT clause ===== is {"*" : doc,}
[2019-12-10 23:15:42,798] - [task:2607] INFO -  <<<<< START Executing Query SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2019-12-10 23:15:42,799] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM default WHERE  job_title = "Sales" 
[2019-12-10 23:15:42,807] - [rest_client:3465] INFO - query params : scan_consistency=request_plus&statement=SELECT+%2A+FROM+default+WHERE++job_title+%3D+%22Sales%22+
[2019-12-10 23:15:42,945] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 120.972497ms
[2019-12-10 23:15:42,966] - [tuq_helper:214] INFO -  Analyzing Actual Result
[2019-12-10 23:15:42,967] - [tuq_helper:216] INFO -  Analyzing Expected Result
[2019-12-10 23:15:42,970] - [task:2617] INFO -  <<<<< Done Executing Query SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2019-12-10 23:15:42,970] - [task:2645] INFO -  <<<<< Done VERIFYING Query SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2019-12-10 23:15:43,972] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employeee5aa6b473c8443a4882e99c6bf403510job_title'
[2019-12-10 23:15:43,979] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employeee5aa6b473c8443a4882e99c6bf403510job_title%27
[2019-12-10 23:15:43,996] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 14.372354ms
[2019-12-10 23:15:43,996] - [tuq_helper:147] INFO - RUN QUERY DROP INDEX default.employeee5aa6b473c8443a4882e99c6bf403510job_title USING GSI
[2019-12-10 23:15:44,003] - [rest_client:3465] INFO - query params : statement=DROP+INDEX+default.employeee5aa6b473c8443a4882e99c6bf403510job_title+USING+GSI
[2019-12-10 23:15:44,050] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 44.144445ms
[2019-12-10 23:15:44,050] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employeee5aa6b473c8443a4882e99c6bf403510job_title'
[2019-12-10 23:15:44,060] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employeee5aa6b473c8443a4882e99c6bf403510job_title%27
[2019-12-10 23:15:44,086] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 20.22755ms
[2019-12-10 23:15:44,126] - [basetestcase:2436] INFO - list of index nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:15:44,129] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:15:44,129] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:15:44,470] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9000: True content: "/opt/build/ns_server/logs/n_0" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))).
[2019-12-10 23:15:44,471] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: zgrep "panic" "/opt/build/ns_server/logs/n_0"/indexer.log* | wc -l
[2019-12-10 23:15:44,530] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:15:44,545] - [basetestcase:2436] INFO - list of kv nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:15:44,548] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:15:44,549] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:15:44,889] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9000: True content: "/opt/build/ns_server/logs/n_0" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))).
[2019-12-10 23:15:44,890] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: zgrep "panic" "/opt/build/ns_server/logs/n_0"/projector.log* | wc -l
[2019-12-10 23:15:44,950] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:15:44,966] - [basetestcase:2436] INFO - list of n1ql nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:15:44,966] - [tuq_helper:516] INFO - CHECK FOR PRIMARY INDEXES
[2019-12-10 23:15:44,966] - [tuq_helper:523] INFO - DROP PRIMARY INDEX ON default USING VIEW 
[2019-12-10 23:15:44,967] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2019-12-10 23:15:44,973] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2019-12-10 23:15:44,986] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 10.370777ms
[2019-12-10 23:15:44,987] - [tuq_helper:147] INFO - RUN QUERY DROP PRIMARY INDEX ON default USING VIEW 
[2019-12-10 23:15:44,993] - [rest_client:3465] INFO - query params : statement=DROP+PRIMARY+INDEX+ON+default+USING+VIEW+
[2019-12-10 23:15:45,007] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 11.059704ms
[2019-12-10 23:15:45,008] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: rm -f /opt/couchbase/var/lib/couchbase/meta;rm -f /tmp/log_upr_client.sock
[2019-12-10 23:15:45,066] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:15:45,067] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: killall cbq-engine
[2019-12-10 23:15:45,129] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:15:45,130] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: killall tuqtng
[2019-12-10 23:15:45,198] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:15:45,199] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: killall indexer
[2019-12-10 23:15:45,278] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:15:45,312] - [basetestcase:397] INFO - ------- Cluster statistics -------
[2019-12-10 23:15:45,312] - [basetestcase:399] INFO - 127.0.0.1:9000 => {'swap_mem_used': 0, 'cpu_utilization': 21.95121951219512, 'mem_free': 13694992384, 'swap_mem_total': 0, 'mem_total': 15466930176, 'services': [u'index', u'kv', u'n1ql']}
[2019-12-10 23:15:45,312] - [basetestcase:400] INFO - --- End of cluster statistics ---
[2019-12-10 23:15:45,313] - [basetestcase:462] INFO - ==============  basetestcase cleanup was started for test #3 test_multi_create_query_explain_drop_index ==============
[2019-12-10 23:15:45,393] - [bucket_helper:143] INFO - deleting existing buckets [u'default'] on 127.0.0.1
[2019-12-10 23:15:45,393] - [bucket_helper:145] INFO - remove bucket default ...
[2019-12-10 23:15:46,361] - [bucket_helper:159] INFO - deleted bucket : default from 127.0.0.1
[2019-12-10 23:15:46,362] - [bucket_helper:237] INFO - waiting for bucket deletion to complete....
[2019-12-10 23:15:46,367] - [rest_client:133] INFO - node 127.0.0.1 existing buckets : []
[2019-12-10 23:15:46,368] - [bucket_helper:169] INFO - sleep 2 seconds to make sure all buckets ([u'default']) were deleted completely.
[2019-12-10 23:15:48,434] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9000
[2019-12-10 23:15:48,440] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9000 is running
[2019-12-10 23:15:48,447] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9001
[2019-12-10 23:15:48,452] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9001 is running
[2019-12-10 23:15:48,459] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9002
[2019-12-10 23:15:48,465] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9002 is running
[2019-12-10 23:15:48,472] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9003
[2019-12-10 23:15:48,478] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9003 is running
[2019-12-10 23:15:48,478] - [basetestcase:480] INFO - ==============  basetestcase cleanup was finished for test #3 test_multi_create_query_explain_drop_index ==============
ok

----------------------------------------------------------------------
Ran 1 test in 30.454s

OK
test_multi_create_query_explain_drop_index (2i.indexscans_2i.SecondaryIndexingScanTests) ... Cluster instance shutdown with force
summary so far suite 2i.indexscans_2i.SecondaryIndexingScanTests , pass 3 , fail 0
testrunner logs, diags and results are available under /opt/build/testrunner/logs/testrunner-19-Dec-10_23-14-02/test_3
Logs will be stored at /opt/build/testrunner/logs/testrunner-19-Dec-10_23-14-02/test_4

./testrunner -i b/resources/dev-4-nodes-xdcr_n1ql_2i.ini -p makefile=True -t 2i.indexscans_2i.SecondaryIndexingScanTests.test_multi_create_query_explain_drop_index,groups=simple:equals:no_orderby_groupby:range,dataset=default,doc-per-day=1,use_gsi_for_primary=False,use_gsi_for_secondary=False,GROUP=views

Test Input params:
{'doc-per-day': '1', 'use_gsi_for_primary': 'False', 'GROUP': 'views', 'use_gsi_for_secondary': 'False', 'gsi_type': 'forestdb', 'makefile': 'True', 'conf_file': 'conf/simple_2i_n1ql.conf', 'dataset': 'default', 'cluster_name': 'dev-4-nodes-xdcr_n1ql_2i', 'ini': 'b/resources/dev-4-nodes-xdcr_n1ql_2i.ini', 'groups': 'simple:equals:no_orderby_groupby:range', 'case_number': 4, 'num_nodes': 4, 'logs_folder': '/opt/build/testrunner/logs/testrunner-19-Dec-10_23-14-02/test_4', 'spec': 'simple_2i_n1ql'}
[2019-12-10 23:15:48,523] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:15:48,523] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:15:48,897] - [rest_client:2127] INFO - Node version in cluster 6.5.0-4928-rel-enterprise
[2019-12-10 23:15:48,911] - [rest_client:2137] INFO - Node versions in cluster [u'6.5.0-4928-rel-enterprise']
[2019-12-10 23:15:48,912] - [basetestcase:218] INFO - ==============  basetestcase setup was started for test #4 test_multi_create_query_explain_drop_index==============
[2019-12-10 23:15:49,003] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9000
[2019-12-10 23:15:49,010] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9000 is running
[2019-12-10 23:15:49,016] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9001
[2019-12-10 23:15:49,022] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9001 is running
[2019-12-10 23:15:49,028] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9002
[2019-12-10 23:15:49,034] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9002 is running
[2019-12-10 23:15:49,041] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9003
[2019-12-10 23:15:49,047] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9003 is running
[2019-12-10 23:15:49,048] - [basetestcase:265] INFO - initializing cluster
[2019-12-10 23:15:49,535] - [task:132] INFO - server: ip:127.0.0.1 port:9000 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_0@cb.local', 'uptime': u'100', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9000', 'storage': [], 'moxi': 11211, 'port': u'9000', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12000, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'index', u'kv', u'n1ql'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13744586752, 'memoryTotal': 15466930176, 'memoryQuota': 7650, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:15:49,544] - [task:171] INFO - quota for index service will be 256 MB
[2019-12-10 23:15:49,544] - [task:173] INFO - set index quota to node 127.0.0.1 
[2019-12-10 23:15:49,544] - [rest_client:965] INFO - pools/default params : indexMemoryQuota=256
[2019-12-10 23:15:49,554] - [rest_client:953] INFO - pools/default params : memoryQuota=7650
[2019-12-10 23:15:49,569] - [rest_client:931] INFO - /node/controller/setupServices params on 127.0.0.1:9000:services=kv%2Cindex%2Cn1ql&password=asdasd&user=Administrator&hostname=127.0.0.1%3A9000
[2019-12-10 23:15:49,579] - [rest_client:840] ERROR - POST http://127.0.0.1:9000//node/controller/setupServices body: services=kv%2Cindex%2Cn1ql&password=asdasd&user=Administrator&hostname=127.0.0.1%3A9000 headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown ["cannot change node services after cluster is provisioned"] auth: Administrator:asdasd
[2019-12-10 23:15:49,584] - [rest_client:936] INFO - This node is already provisioned with services, we do not consider this as failure for test case
[2019-12-10 23:15:49,585] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9000:username=Administrator&password=asdasd&port=9000
[2019-12-10 23:15:49,644] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:15:49,644] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:15:49,985] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9000/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:15:50,049] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:15:50,057] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9000: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:15:50,058] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:15:50,064] - [rest_client:840] ERROR - POST http://127.0.0.1:9000/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:15:50,092] - [task:132] INFO - server: ip:127.0.0.1 port:9001 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_1@cb.local', 'uptime': u'104', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9001', 'storage': [], 'moxi': 11211, 'port': u'9001', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12002, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13766742016, 'memoryTotal': 15466930176, 'memoryQuota': 7906, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:15:50,102] - [rest_client:953] INFO - pools/default params : memoryQuota=7906
[2019-12-10 23:15:50,110] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9001:username=Administrator&password=asdasd&port=9001
[2019-12-10 23:15:50,175] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:15:50,176] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:15:50,521] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9001/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:15:50,586] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:15:50,594] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9001: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:15:50,594] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:15:50,599] - [rest_client:840] ERROR - POST http://127.0.0.1:9001/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:15:50,629] - [task:132] INFO - server: ip:127.0.0.1 port:9002 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_2@cb.local', 'uptime': u'104', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9002', 'storage': [], 'moxi': 11211, 'port': u'9002', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12004, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13766602752, 'memoryTotal': 15466930176, 'memoryQuota': 7906, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:15:50,640] - [rest_client:953] INFO - pools/default params : memoryQuota=7906
[2019-12-10 23:15:50,648] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9002:username=Administrator&password=asdasd&port=9002
[2019-12-10 23:15:50,709] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:15:50,710] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:15:51,080] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9002/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:15:51,147] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:15:51,155] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9002: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:15:51,155] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:15:51,161] - [rest_client:840] ERROR - POST http://127.0.0.1:9002/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:15:51,188] - [task:132] INFO - server: ip:127.0.0.1 port:9003 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_3@cb.local', 'uptime': u'104', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9003', 'storage': [], 'moxi': 11211, 'port': u'9003', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12006, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13767471104, 'memoryTotal': 15466930176, 'memoryQuota': 7906, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:15:51,197] - [rest_client:953] INFO - pools/default params : memoryQuota=7906
[2019-12-10 23:15:51,206] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9003:username=Administrator&password=asdasd&port=9003
[2019-12-10 23:15:51,266] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:15:51,267] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:15:51,619] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9003/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:15:51,684] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:15:51,691] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9003: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:15:51,692] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:15:51,698] - [rest_client:840] ERROR - POST http://127.0.0.1:9003/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:15:51,730] - [basetestcase:2218] INFO - **** add built-in 'cbadminbucket' user to node 127.0.0.1 ****
[2019-12-10 23:15:51,833] - [basetestcase:530] INFO - sleep for 5 secs.  ...
[2019-12-10 23:15:56,838] - [basetestcase:2223] INFO - **** add 'admin' role to 'cbadminbucket' user ****
[2019-12-10 23:15:56,861] - [basetestcase:291] INFO - done initializing cluster
[2019-12-10 23:15:56,865] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:15:56,866] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:15:57,251] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9000/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:15:57,322] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:15:57,324] - [basetestcase:2770] INFO - Enabled diag/eval for non-local hosts from 127.0.0.1
[2019-12-10 23:15:57,752] - [rest_client:2430] INFO - http://127.0.0.1:9000/pools/default/buckets with param: bucketType=membase&evictionPolicy=valueOnly&threadsNumber=3&ramQuotaMB=7650&authType=sasl&name=default&flushEnabled=1&replicaNumber=1&compressionMode=passive&saslPassword=None&replicaIndex=1
[2019-12-10 23:15:57,771] - [rest_client:2455] INFO - 0.02 seconds to create bucket default
[2019-12-10 23:15:57,772] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:15:57,969] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:15:58,059] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:15:58,114] - [task:351] WARNING - vbucket map not ready after try 0
[2019-12-10 23:15:58,115] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:15:58,219] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:15:58,302] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:15:58,346] - [task:351] WARNING - vbucket map not ready after try 1
[2019-12-10 23:15:58,347] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:15:58,450] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:15:58,537] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:15:58,580] - [task:351] WARNING - vbucket map not ready after try 2
[2019-12-10 23:15:58,581] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:15:58,693] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:15:58,778] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:15:58,825] - [task:351] WARNING - vbucket map not ready after try 3
[2019-12-10 23:15:58,825] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:15:58,939] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:15:59,054] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:15:59,098] - [task:351] WARNING - vbucket map not ready after try 4
[2019-12-10 23:15:59,098] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:15:59,178] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:15:59,286] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:15:59,329] - [task:351] WARNING - vbucket map not ready after try 5
[2019-12-10 23:15:59,381] - [basetestcase:383] INFO - ==============  basetestcase setup was finished for test #4 test_multi_create_query_explain_drop_index ==============
[2019-12-10 23:15:59,405] - [basetestcase:397] INFO - ------- Cluster statistics -------
[2019-12-10 23:15:59,405] - [basetestcase:399] INFO - 127.0.0.1:9000 => {'swap_mem_used': 0, 'cpu_utilization': 32.71767810026385, 'mem_free': 13766819840, 'swap_mem_total': 0, 'mem_total': 15466930176, 'services': [u'index', u'kv', u'n1ql']}
[2019-12-10 23:15:59,405] - [basetestcase:400] INFO - --- End of cluster statistics ---
[2019-12-10 23:15:59,417] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is warmup
[2019-12-10 23:15:59,418] - [newtuq:22] INFO - Waiting for cluster to become healthy
[2019-12-10 23:15:59,418] - [basetestcase:530] INFO - sleep for 5 secs.  ...
[2019-12-10 23:16:04,431] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:16:04,444] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is healthy
[2019-12-10 23:16:04,444] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:16:04,458] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is healthy
[2019-12-10 23:16:04,458] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:16:04,471] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is healthy
[2019-12-10 23:16:04,472] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:16:04,483] - [basetestcase:2436] INFO - list of index nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:16:04,508] - [rest_client:1706] INFO - {'indexer.settings.storage_mode': 'forestdb'} set
[2019-12-10 23:16:04,537] - [rest_client:1706] INFO - {'indexer.api.enableTestServer': True} set
[2019-12-10 23:16:04,540] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:16:04,540] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:16:06,055] - [basetestcase:2545] INFO - create 2016 to default documents...
[2019-12-10 23:16:06,111] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:16:07,567] - [basetestcase:2557] INFO - LOAD IS FINISHED
[2019-12-10 23:16:07,609] - [basetestcase:2436] INFO - list of n1ql nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:16:07,610] - [newtuq:83] INFO - ip:127.0.0.1 port:9000 ssh_username:Administrator
[2019-12-10 23:16:07,610] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2019-12-10 23:16:07,618] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2019-12-10 23:16:07,713] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 91.575501ms
[2019-12-10 23:16:07,713] - [tuq_helper:147] INFO - RUN QUERY CREATE PRIMARY INDEX ON default  USING VIEW 
[2019-12-10 23:16:07,722] - [rest_client:3465] INFO - query params : statement=CREATE+PRIMARY+INDEX+ON+default++USING+VIEW+
[2019-12-10 23:16:08,224] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 499.457498ms
[2019-12-10 23:16:08,225] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2019-12-10 23:16:08,231] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2019-12-10 23:16:08,245] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 10.856394ms
[2019-12-10 23:16:08,271] - [basetestcase:2436] INFO - list of index nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:16:09,260] - [tuq_helper:147] INFO - RUN QUERY CREATE INDEX employeedc58bcbc032b487c9c3c7d540b00bbc2job_title ON default(job_title) USING VIEW 
[2019-12-10 23:16:09,267] - [rest_client:3465] INFO - query params : statement=CREATE+INDEX+employeedc58bcbc032b487c9c3c7d540b00bbc2job_title+ON+default%28job_title%29+USING+VIEW+
[2019-12-10 23:16:09,746] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 476.100789ms
[2019-12-10 23:16:09,746] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employeedc58bcbc032b487c9c3c7d540b00bbc2job_title'
[2019-12-10 23:16:09,753] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employeedc58bcbc032b487c9c3c7d540b00bbc2job_title%27
[2019-12-10 23:16:09,771] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 15.752912ms
[2019-12-10 23:16:10,772] - [task:2607] INFO -  <<<<< START Executing Query EXPLAIN SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2019-12-10 23:16:10,773] - [tuq_helper:147] INFO - RUN QUERY EXPLAIN SELECT * FROM default WHERE  job_title = "Sales" 
[2019-12-10 23:16:10,779] - [rest_client:3465] INFO - query params : statement=EXPLAIN+SELECT+%2A+FROM+default+WHERE++job_title+%3D+%22Sales%22+
[2019-12-10 23:16:10,785] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 3.131641ms
[2019-12-10 23:16:10,785] - [task:2616] INFO - {u'status': u'success', u'metrics': {u'elapsedTime': u'3.131641ms', u'executionTime': u'3.07648ms', u'resultSize': 722, u'resultCount': 1}, u'results': [{u'text': u'SELECT * FROM default WHERE  job_title = "Sales"', u'plan': {u'#operator': u'Sequence', u'~children': [{u'index': u'employeedc58bcbc032b487c9c3c7d540b00bbc2job_title', u'namespace': u'default', u'keyspace': u'default', u'#operator': u'IndexScan', u'spans': [{u'Range': {u'High': [u'"Sales"'], u'Low': [u'"Sales"'], u'Inclusion': 3}, u'Exact': True}], u'using': u'view', u'index_id': u'employeedc58bcbc032b487c9c3c7d540b00bbc2job_title'}, {u'keyspace': u'default', u'#operator': u'Fetch', u'namespace': u'default'}, {u'#operator': u'Parallel', u'~child': {u'#operator': u'Sequence', u'~children': [{u'#operator': u'Filter', u'condition': u'((`default`.`job_title`) = "Sales")'}, {u'#operator': u'InitialProject', u'result_terms': [{u'expr': u'self', u'star': True}]}, {u'#operator': u'FinalProject'}]}}]}}], u'requestID': u'129f3bb4-25ad-41fb-b337-38e71b355022', u'signature': u'json'}
[2019-12-10 23:16:10,785] - [task:2617] INFO -  <<<<< Done Executing Query EXPLAIN SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2019-12-10 23:16:10,786] - [task:2645] INFO -  <<<<< Done VERIFYING Query EXPLAIN SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2019-12-10 23:16:10,786] - [base_2i:314] INFO - Query : SELECT * FROM default WHERE  job_title = "Sales" 
[2019-12-10 23:16:10,786] - [tuq_generators:70] INFO - FROM clause ===== is default
[2019-12-10 23:16:10,787] - [tuq_generators:72] INFO - WHERE clause ===== is   doc["job_title"] == "Sales" 
[2019-12-10 23:16:10,787] - [tuq_generators:74] INFO - UNNEST clause ===== is None
[2019-12-10 23:16:10,787] - [tuq_generators:76] INFO - SELECT clause ===== is {"*" : doc,}
[2019-12-10 23:16:11,787] - [task:2607] INFO -  <<<<< START Executing Query SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2019-12-10 23:16:11,787] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM default WHERE  job_title = "Sales" 
[2019-12-10 23:16:11,794] - [rest_client:3465] INFO - query params : scan_consistency=request_plus&statement=SELECT+%2A+FROM+default+WHERE++job_title+%3D+%22Sales%22+
[2019-12-10 23:16:11,942] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 132.65165ms
[2019-12-10 23:16:11,961] - [tuq_helper:214] INFO -  Analyzing Actual Result
[2019-12-10 23:16:11,961] - [tuq_helper:216] INFO -  Analyzing Expected Result
[2019-12-10 23:16:11,964] - [task:2617] INFO -  <<<<< Done Executing Query SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2019-12-10 23:16:11,964] - [task:2645] INFO -  <<<<< Done VERIFYING Query SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2019-12-10 23:16:12,966] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employeedc58bcbc032b487c9c3c7d540b00bbc2job_title'
[2019-12-10 23:16:12,973] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employeedc58bcbc032b487c9c3c7d540b00bbc2job_title%27
[2019-12-10 23:16:12,992] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 16.347196ms
[2019-12-10 23:16:12,992] - [tuq_helper:147] INFO - RUN QUERY DROP INDEX default.employeedc58bcbc032b487c9c3c7d540b00bbc2job_title USING VIEW 
[2019-12-10 23:16:12,998] - [rest_client:3465] INFO - query params : statement=DROP+INDEX+default.employeedc58bcbc032b487c9c3c7d540b00bbc2job_title+USING+VIEW+
[2019-12-10 23:16:13,016] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 14.504784ms
[2019-12-10 23:16:13,017] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employeedc58bcbc032b487c9c3c7d540b00bbc2job_title'
[2019-12-10 23:16:13,028] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employeedc58bcbc032b487c9c3c7d540b00bbc2job_title%27
[2019-12-10 23:16:13,041] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 11.0928ms
[2019-12-10 23:16:13,067] - [basetestcase:2436] INFO - list of index nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:16:13,070] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:16:13,070] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:16:13,458] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9000: True content: "/opt/build/ns_server/logs/n_0" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))).
[2019-12-10 23:16:13,459] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: zgrep "panic" "/opt/build/ns_server/logs/n_0"/indexer.log* | wc -l
[2019-12-10 23:16:13,525] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:16:13,540] - [basetestcase:2436] INFO - list of kv nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:16:13,544] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:16:13,544] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:16:13,923] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9000: True content: "/opt/build/ns_server/logs/n_0" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))).
[2019-12-10 23:16:13,924] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: zgrep "panic" "/opt/build/ns_server/logs/n_0"/projector.log* | wc -l
[2019-12-10 23:16:13,987] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:16:14,002] - [basetestcase:2436] INFO - list of n1ql nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:16:14,003] - [tuq_helper:516] INFO - CHECK FOR PRIMARY INDEXES
[2019-12-10 23:16:14,003] - [tuq_helper:523] INFO - DROP PRIMARY INDEX ON default USING VIEW 
[2019-12-10 23:16:14,003] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2019-12-10 23:16:14,010] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2019-12-10 23:16:14,027] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 14.874087ms
[2019-12-10 23:16:14,027] - [tuq_helper:147] INFO - RUN QUERY DROP PRIMARY INDEX ON default USING VIEW 
[2019-12-10 23:16:14,034] - [rest_client:3465] INFO - query params : statement=DROP+PRIMARY+INDEX+ON+default+USING+VIEW+
[2019-12-10 23:16:14,087] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 50.400084ms
[2019-12-10 23:16:14,088] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: rm -f /opt/couchbase/var/lib/couchbase/meta;rm -f /tmp/log_upr_client.sock
[2019-12-10 23:16:14,148] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:16:14,149] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: killall cbq-engine
[2019-12-10 23:16:14,209] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:16:14,210] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: killall tuqtng
[2019-12-10 23:16:14,285] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:16:14,287] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: killall indexer
[2019-12-10 23:16:14,366] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:16:14,394] - [basetestcase:397] INFO - ------- Cluster statistics -------
[2019-12-10 23:16:14,395] - [basetestcase:399] INFO - 127.0.0.1:9000 => {'swap_mem_used': 0, 'cpu_utilization': 10.12987012987013, 'mem_free': 13698703360, 'swap_mem_total': 0, 'mem_total': 15466930176, 'services': [u'index', u'kv', u'n1ql']}
[2019-12-10 23:16:14,395] - [basetestcase:400] INFO - --- End of cluster statistics ---
[2019-12-10 23:16:14,395] - [basetestcase:462] INFO - ==============  basetestcase cleanup was started for test #4 test_multi_create_query_explain_drop_index ==============
[2019-12-10 23:16:14,469] - [bucket_helper:143] INFO - deleting existing buckets [u'default'] on 127.0.0.1
[2019-12-10 23:16:14,469] - [bucket_helper:145] INFO - remove bucket default ...
[2019-12-10 23:16:14,833] - [bucket_helper:159] INFO - deleted bucket : default from 127.0.0.1
[2019-12-10 23:16:14,833] - [bucket_helper:237] INFO - waiting for bucket deletion to complete....
[2019-12-10 23:16:14,840] - [rest_client:133] INFO - node 127.0.0.1 existing buckets : []
[2019-12-10 23:16:14,841] - [bucket_helper:169] INFO - sleep 2 seconds to make sure all buckets ([u'default']) were deleted completely.
[2019-12-10 23:16:16,910] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9000
[2019-12-10 23:16:16,916] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9000 is running
[2019-12-10 23:16:16,923] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9001
[2019-12-10 23:16:16,929] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9001 is running
[2019-12-10 23:16:16,936] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9002
[2019-12-10 23:16:16,942] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9002 is running
[2019-12-10 23:16:16,949] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9003
[2019-12-10 23:16:16,956] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9003 is running
[2019-12-10 23:16:16,956] - [basetestcase:480] INFO - ==============  basetestcase cleanup was finished for test #4 test_multi_create_query_explain_drop_index ==============
ok

----------------------------------------------------------------------
Ran 1 test in 28.449s

OK
test_multi_create_query_explain_drop_index (2i.indexscans_2i.SecondaryIndexingScanTests) ... Cluster instance shutdown with force
summary so far suite 2i.indexscans_2i.SecondaryIndexingScanTests , pass 4 , fail 0
testrunner logs, diags and results are available under /opt/build/testrunner/logs/testrunner-19-Dec-10_23-14-02/test_4
Logs will be stored at /opt/build/testrunner/logs/testrunner-19-Dec-10_23-14-02/test_5

./testrunner -i b/resources/dev-4-nodes-xdcr_n1ql_2i.ini -p makefile=True -t 2i.indexscans_2i.SecondaryIndexingScanTests.test_multi_create_query_explain_drop_index,groups=simple:equals:no_orderby_groupby:range,dataset=default,doc-per-day=1,use_gsi_for_primary=True,use_gsi_for_secondary=True,doc_ops=True,delete_ops_per=.5,run_async=True,scan_consistency=request_plus,GROUP=gsi

Test Input params:
{'doc-per-day': '1', 'use_gsi_for_primary': 'True', 'GROUP': 'gsi', 'use_gsi_for_secondary': 'True', 'gsi_type': 'forestdb', 'doc_ops': 'True', 'makefile': 'True', 'delete_ops_per': '.5', 'conf_file': 'conf/simple_2i_n1ql.conf', 'dataset': 'default', 'cluster_name': 'dev-4-nodes-xdcr_n1ql_2i', 'num_nodes': 4, 'ini': 'b/resources/dev-4-nodes-xdcr_n1ql_2i.ini', 'groups': 'simple:equals:no_orderby_groupby:range', 'case_number': 5, 'scan_consistency': 'request_plus', 'run_async': 'True', 'logs_folder': '/opt/build/testrunner/logs/testrunner-19-Dec-10_23-14-02/test_5', 'spec': 'simple_2i_n1ql'}
[2019-12-10 23:16:17,001] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:16:17,001] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:16:17,414] - [rest_client:2127] INFO - Node version in cluster 6.5.0-4928-rel-enterprise
[2019-12-10 23:16:17,427] - [rest_client:2137] INFO - Node versions in cluster [u'6.5.0-4928-rel-enterprise']
[2019-12-10 23:16:17,428] - [basetestcase:218] INFO - ==============  basetestcase setup was started for test #5 test_multi_create_query_explain_drop_index==============
[2019-12-10 23:16:17,521] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9000
[2019-12-10 23:16:17,527] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9000 is running
[2019-12-10 23:16:17,534] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9001
[2019-12-10 23:16:17,541] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9001 is running
[2019-12-10 23:16:17,547] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9002
[2019-12-10 23:16:17,553] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9002 is running
[2019-12-10 23:16:17,560] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9003
[2019-12-10 23:16:17,566] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9003 is running
[2019-12-10 23:16:17,567] - [basetestcase:265] INFO - initializing cluster
[2019-12-10 23:16:18,012] - [task:132] INFO - server: ip:127.0.0.1 port:9000 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_0@cb.local', 'uptime': u'129', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9000', 'storage': [], 'moxi': 11211, 'port': u'9000', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12000, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'index', u'kv', u'n1ql'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13725556736, 'memoryTotal': 15466930176, 'memoryQuota': 7650, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:16:18,018] - [task:171] INFO - quota for index service will be 256 MB
[2019-12-10 23:16:18,019] - [task:173] INFO - set index quota to node 127.0.0.1 
[2019-12-10 23:16:18,019] - [rest_client:965] INFO - pools/default params : indexMemoryQuota=256
[2019-12-10 23:16:18,043] - [rest_client:953] INFO - pools/default params : memoryQuota=7650
[2019-12-10 23:16:18,054] - [rest_client:931] INFO - /node/controller/setupServices params on 127.0.0.1:9000:services=kv%2Cindex%2Cn1ql&password=asdasd&user=Administrator&hostname=127.0.0.1%3A9000
[2019-12-10 23:16:18,061] - [rest_client:840] ERROR - POST http://127.0.0.1:9000//node/controller/setupServices body: services=kv%2Cindex%2Cn1ql&password=asdasd&user=Administrator&hostname=127.0.0.1%3A9000 headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown ["cannot change node services after cluster is provisioned"] auth: Administrator:asdasd
[2019-12-10 23:16:18,066] - [rest_client:936] INFO - This node is already provisioned with services, we do not consider this as failure for test case
[2019-12-10 23:16:18,066] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9000:username=Administrator&password=asdasd&port=9000
[2019-12-10 23:16:18,128] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:16:18,128] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:16:18,545] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9000/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:16:18,614] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:16:18,622] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9000: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:16:18,623] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:16:18,628] - [rest_client:840] ERROR - POST http://127.0.0.1:9000/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:16:18,656] - [task:132] INFO - server: ip:127.0.0.1 port:9001 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_1@cb.local', 'uptime': u'129', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9001', 'storage': [], 'moxi': 11211, 'port': u'9001', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12002, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13706133504, 'memoryTotal': 15466930176, 'memoryQuota': 7906, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:16:18,666] - [rest_client:953] INFO - pools/default params : memoryQuota=7906
[2019-12-10 23:16:18,675] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9001:username=Administrator&password=asdasd&port=9001
[2019-12-10 23:16:18,731] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:16:18,731] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:16:19,126] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9001/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:16:19,195] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:16:19,204] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9001: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:16:19,204] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:16:19,209] - [rest_client:840] ERROR - POST http://127.0.0.1:9001/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:16:19,235] - [task:132] INFO - server: ip:127.0.0.1 port:9002 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_2@cb.local', 'uptime': u'129', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9002', 'storage': [], 'moxi': 11211, 'port': u'9002', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12004, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13702217728, 'memoryTotal': 15466930176, 'memoryQuota': 7906, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:16:19,245] - [rest_client:953] INFO - pools/default params : memoryQuota=7906
[2019-12-10 23:16:19,253] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9002:username=Administrator&password=asdasd&port=9002
[2019-12-10 23:16:19,311] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:16:19,311] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:16:19,700] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9002/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:16:19,771] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:16:19,783] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9002: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:16:19,784] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:16:19,790] - [rest_client:840] ERROR - POST http://127.0.0.1:9002/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:16:19,818] - [task:132] INFO - server: ip:127.0.0.1 port:9003 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_3@cb.local', 'uptime': u'129', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9003', 'storage': [], 'moxi': 11211, 'port': u'9003', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12006, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13700902912, 'memoryTotal': 15466930176, 'memoryQuota': 7906, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:16:19,828] - [rest_client:953] INFO - pools/default params : memoryQuota=7906
[2019-12-10 23:16:19,838] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9003:username=Administrator&password=asdasd&port=9003
[2019-12-10 23:16:19,892] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:16:19,893] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:16:20,309] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9003/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:16:20,379] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:16:20,386] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9003: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:16:20,386] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:16:20,392] - [rest_client:840] ERROR - POST http://127.0.0.1:9003/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:16:20,423] - [basetestcase:2218] INFO - **** add built-in 'cbadminbucket' user to node 127.0.0.1 ****
[2019-12-10 23:16:20,530] - [basetestcase:530] INFO - sleep for 5 secs.  ...
[2019-12-10 23:16:25,535] - [basetestcase:2223] INFO - **** add 'admin' role to 'cbadminbucket' user ****
[2019-12-10 23:16:25,560] - [basetestcase:291] INFO - done initializing cluster
[2019-12-10 23:16:25,564] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:16:25,564] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:16:25,984] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9000/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:16:26,056] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:16:26,057] - [basetestcase:2770] INFO - Enabled diag/eval for non-local hosts from 127.0.0.1
[2019-12-10 23:16:26,449] - [rest_client:2430] INFO - http://127.0.0.1:9000/pools/default/buckets with param: bucketType=membase&evictionPolicy=valueOnly&threadsNumber=3&ramQuotaMB=7650&authType=sasl&name=default&flushEnabled=1&replicaNumber=1&compressionMode=passive&saslPassword=None&replicaIndex=1
[2019-12-10 23:16:26,460] - [rest_client:2455] INFO - 0.01 seconds to create bucket default
[2019-12-10 23:16:26,462] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:16:26,612] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:16:26,770] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:16:26,821] - [task:351] WARNING - vbucket map not ready after try 0
[2019-12-10 23:16:26,821] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:16:26,900] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:16:27,013] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:16:27,056] - [task:351] WARNING - vbucket map not ready after try 1
[2019-12-10 23:16:27,056] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:16:27,138] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:16:27,261] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:16:27,318] - [task:351] WARNING - vbucket map not ready after try 2
[2019-12-10 23:16:27,319] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:16:27,397] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:16:27,517] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:16:27,577] - [task:351] WARNING - vbucket map not ready after try 3
[2019-12-10 23:16:27,577] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:16:27,681] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:16:27,767] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:16:27,844] - [task:351] WARNING - vbucket map not ready after try 4
[2019-12-10 23:16:27,844] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:16:27,925] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:16:28,004] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:16:28,079] - [task:351] WARNING - vbucket map not ready after try 5
[2019-12-10 23:16:28,112] - [basetestcase:383] INFO - ==============  basetestcase setup was finished for test #5 test_multi_create_query_explain_drop_index ==============
[2019-12-10 23:16:28,135] - [basetestcase:397] INFO - ------- Cluster statistics -------
[2019-12-10 23:16:28,135] - [basetestcase:399] INFO - 127.0.0.1:9000 => {'swap_mem_used': 0, 'cpu_utilization': 22.05128205128205, 'mem_free': 13742796800, 'swap_mem_total': 0, 'mem_total': 15466930176, 'services': [u'index', u'kv', u'n1ql']}
[2019-12-10 23:16:28,135] - [basetestcase:400] INFO - --- End of cluster statistics ---
[2019-12-10 23:16:28,147] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is warmup
[2019-12-10 23:16:28,147] - [newtuq:22] INFO - Waiting for cluster to become healthy
[2019-12-10 23:16:28,148] - [basetestcase:530] INFO - sleep for 5 secs.  ...
[2019-12-10 23:16:33,161] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:16:33,175] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is healthy
[2019-12-10 23:16:33,175] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:16:33,189] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is healthy
[2019-12-10 23:16:33,189] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:16:33,202] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is healthy
[2019-12-10 23:16:33,203] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:16:33,216] - [basetestcase:2436] INFO - list of index nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:16:33,234] - [rest_client:1706] INFO - {'indexer.settings.storage_mode': 'forestdb'} set
[2019-12-10 23:16:33,265] - [rest_client:1706] INFO - {'indexer.api.enableTestServer': True} set
[2019-12-10 23:16:33,269] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:16:33,269] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:16:34,771] - [basetestcase:2545] INFO - create 2016 to default documents...
[2019-12-10 23:16:34,828] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:16:36,343] - [basetestcase:2557] INFO - LOAD IS FINISHED
[2019-12-10 23:16:36,365] - [newtuq:73] INFO - {'remaining': {'start': 0, 'end': 1}, 'delete': {'start': 0, 'end': 0}}
[2019-12-10 23:16:37,028] - [basetestcase:2436] INFO - list of n1ql nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:16:37,028] - [newtuq:83] INFO - ip:127.0.0.1 port:9000 ssh_username:Administrator
[2019-12-10 23:16:37,028] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2019-12-10 23:16:37,035] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2019-12-10 23:16:37,124] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 86.194697ms
[2019-12-10 23:16:37,124] - [tuq_helper:147] INFO - RUN QUERY CREATE PRIMARY INDEX ON default  USING GSI
[2019-12-10 23:16:37,132] - [rest_client:3465] INFO - query params : statement=CREATE+PRIMARY+INDEX+ON+default++USING+GSI
[2019-12-10 23:16:37,797] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 661.099441ms
[2019-12-10 23:16:37,797] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2019-12-10 23:16:37,808] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2019-12-10 23:16:37,835] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 21.557982ms
[2019-12-10 23:16:37,877] - [basetestcase:2436] INFO - list of index nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:16:38,861] - [tuq_helper:147] INFO - RUN QUERY CREATE INDEX employee2ce271ed4e984e3c8edd5eebd6e32459job_title ON default(job_title) USING GSI  WITH {'nodes': ['127.0.0.1:9000'], 'defer_build': True}
[2019-12-10 23:16:38,868] - [rest_client:3465] INFO - query params : statement=CREATE+INDEX+employee2ce271ed4e984e3c8edd5eebd6e32459job_title+ON+default%28job_title%29+USING+GSI++WITH+%7B%27nodes%27%3A+%5B%27127.0.0.1%3A9000%27%5D%2C+%27defer_build%27%3A+True%7D
[2019-12-10 23:16:39,107] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 236.521993ms
[2019-12-10 23:16:39,108] - [base_2i:109] INFO - BUILD INDEX on default(employee2ce271ed4e984e3c8edd5eebd6e32459job_title) USING GSI
[2019-12-10 23:16:40,108] - [tuq_helper:147] INFO - RUN QUERY BUILD INDEX on default(employee2ce271ed4e984e3c8edd5eebd6e32459job_title) USING GSI
[2019-12-10 23:16:40,116] - [rest_client:3465] INFO - query params : statement=BUILD+INDEX+on+default%28employee2ce271ed4e984e3c8edd5eebd6e32459job_title%29+USING+GSI
[2019-12-10 23:16:40,187] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 67.727559ms
[2019-12-10 23:16:41,189] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee2ce271ed4e984e3c8edd5eebd6e32459job_title'
[2019-12-10 23:16:41,197] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee2ce271ed4e984e3c8edd5eebd6e32459job_title%27
[2019-12-10 23:16:41,210] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 11.181369ms
[2019-12-10 23:16:42,212] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee2ce271ed4e984e3c8edd5eebd6e32459job_title'
[2019-12-10 23:16:42,219] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee2ce271ed4e984e3c8edd5eebd6e32459job_title%27
[2019-12-10 23:16:42,233] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 11.050499ms
[2019-12-10 23:16:42,951] - [basetestcase:2545] INFO - delete 0 to default documents...
[2019-12-10 23:16:43,012] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:16:44,212] - [basetestcase:2557] INFO - LOAD IS FINISHED
[2019-12-10 23:16:45,241] - [task:2607] INFO -  <<<<< START Executing Query EXPLAIN SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2019-12-10 23:16:45,241] - [tuq_helper:147] INFO - RUN QUERY EXPLAIN SELECT * FROM default WHERE  job_title = "Sales" 
[2019-12-10 23:16:45,252] - [rest_client:3465] INFO - query params : statement=EXPLAIN+SELECT+%2A+FROM+default+WHERE++job_title+%3D+%22Sales%22+
[2019-12-10 23:16:45,258] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 3.864401ms
[2019-12-10 23:16:45,258] - [task:2616] INFO - {u'status': u'success', u'metrics': {u'elapsedTime': u'3.864401ms', u'executionTime': u'3.758498ms', u'resultSize': 727, u'resultCount': 1}, u'results': [{u'text': u'SELECT * FROM default WHERE  job_title = "Sales"', u'plan': {u'#operator': u'Sequence', u'~children': [{u'index': u'employee2ce271ed4e984e3c8edd5eebd6e32459job_title', u'namespace': u'default', u'keyspace': u'default', u'#operator': u'IndexScan3', u'index_projection': {u'primary_key': True}, u'spans': [{u'range': [{u'high': u'"Sales"', u'low': u'"Sales"', u'inclusion': 3}], u'exact': True}], u'using': u'gsi', u'index_id': u'27b4f2fc028dd9de'}, {u'keyspace': u'default', u'#operator': u'Fetch', u'namespace': u'default'}, {u'#operator': u'Parallel', u'~child': {u'#operator': u'Sequence', u'~children': [{u'#operator': u'Filter', u'condition': u'((`default`.`job_title`) = "Sales")'}, {u'#operator': u'InitialProject', u'result_terms': [{u'expr': u'self', u'star': True}]}, {u'#operator': u'FinalProject'}]}}]}}], u'requestID': u'f1d595ce-8296-495c-8340-7cf3a3bbb6db', u'signature': u'json'}
[2019-12-10 23:16:45,259] - [task:2617] INFO -  <<<<< Done Executing Query EXPLAIN SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2019-12-10 23:16:45,259] - [task:2645] INFO -  <<<<< Done VERIFYING Query EXPLAIN SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2019-12-10 23:16:45,260] - [base_2i:314] INFO - Query : SELECT * FROM default WHERE  job_title = "Sales" 
[2019-12-10 23:16:45,260] - [tuq_generators:70] INFO - FROM clause ===== is default
[2019-12-10 23:16:45,261] - [tuq_generators:72] INFO - WHERE clause ===== is   doc["job_title"] == "Sales" 
[2019-12-10 23:16:45,261] - [tuq_generators:74] INFO - UNNEST clause ===== is None
[2019-12-10 23:16:45,261] - [tuq_generators:76] INFO - SELECT clause ===== is {"*" : doc,}
[2019-12-10 23:16:46,261] - [task:2607] INFO -  <<<<< START Executing Query SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2019-12-10 23:16:46,261] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM default WHERE  job_title = "Sales" 
[2019-12-10 23:16:46,268] - [rest_client:3465] INFO - query params : scan_consistency=request_plus&statement=SELECT+%2A+FROM+default+WHERE++job_title+%3D+%22Sales%22+
[2019-12-10 23:16:46,417] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 125.967621ms
[2019-12-10 23:16:46,438] - [tuq_helper:214] INFO -  Analyzing Actual Result
[2019-12-10 23:16:46,438] - [tuq_helper:216] INFO -  Analyzing Expected Result
[2019-12-10 23:16:46,441] - [task:2617] INFO -  <<<<< Done Executing Query SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2019-12-10 23:16:46,442] - [task:2645] INFO -  <<<<< Done VERIFYING Query SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2019-12-10 23:16:47,443] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee2ce271ed4e984e3c8edd5eebd6e32459job_title'
[2019-12-10 23:16:47,450] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee2ce271ed4e984e3c8edd5eebd6e32459job_title%27
[2019-12-10 23:16:47,465] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 11.969072ms
[2019-12-10 23:16:47,465] - [tuq_helper:147] INFO - RUN QUERY DROP INDEX default.employee2ce271ed4e984e3c8edd5eebd6e32459job_title USING GSI
[2019-12-10 23:16:47,471] - [rest_client:3465] INFO - query params : statement=DROP+INDEX+default.employee2ce271ed4e984e3c8edd5eebd6e32459job_title+USING+GSI
[2019-12-10 23:16:47,527] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 52.503908ms
[2019-12-10 23:16:47,528] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee2ce271ed4e984e3c8edd5eebd6e32459job_title'
[2019-12-10 23:16:47,538] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee2ce271ed4e984e3c8edd5eebd6e32459job_title%27
[2019-12-10 23:16:47,556] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 15.073073ms
[2019-12-10 23:16:47,593] - [basetestcase:2436] INFO - list of index nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:16:47,598] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:16:47,599] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:16:48,158] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9000: True content: "/opt/build/ns_server/logs/n_0" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))).
[2019-12-10 23:16:48,158] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: zgrep "panic" "/opt/build/ns_server/logs/n_0"/indexer.log* | wc -l
[2019-12-10 23:16:48,246] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:16:48,265] - [basetestcase:2436] INFO - list of kv nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:16:48,270] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:16:48,270] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:16:48,827] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9000: True content: "/opt/build/ns_server/logs/n_0" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))).
[2019-12-10 23:16:48,828] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: zgrep "panic" "/opt/build/ns_server/logs/n_0"/projector.log* | wc -l
[2019-12-10 23:16:48,924] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:16:48,943] - [basetestcase:2436] INFO - list of n1ql nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:16:48,943] - [tuq_helper:516] INFO - CHECK FOR PRIMARY INDEXES
[2019-12-10 23:16:48,943] - [tuq_helper:523] INFO - DROP PRIMARY INDEX ON default USING GSI
[2019-12-10 23:16:48,944] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2019-12-10 23:16:48,954] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2019-12-10 23:16:48,968] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 11.263413ms
[2019-12-10 23:16:48,969] - [tuq_helper:147] INFO - RUN QUERY DROP PRIMARY INDEX ON default USING GSI
[2019-12-10 23:16:48,976] - [rest_client:3465] INFO - query params : statement=DROP+PRIMARY+INDEX+ON+default+USING+GSI
[2019-12-10 23:16:49,021] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 41.966144ms
[2019-12-10 23:16:49,022] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: rm -f /opt/couchbase/var/lib/couchbase/meta;rm -f /tmp/log_upr_client.sock
[2019-12-10 23:16:49,110] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:16:49,111] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: killall cbq-engine
[2019-12-10 23:16:49,184] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:16:49,185] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: killall tuqtng
[2019-12-10 23:16:49,283] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:16:49,284] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: killall indexer
[2019-12-10 23:16:49,388] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:16:49,430] - [basetestcase:397] INFO - ------- Cluster statistics -------
[2019-12-10 23:16:49,430] - [basetestcase:399] INFO - 127.0.0.1:9000 => {'swap_mem_used': 0, 'cpu_utilization': 7.08215297450425, 'mem_free': 13550313472, 'swap_mem_total': 0, 'mem_total': 15466930176, 'services': [u'index', u'kv', u'n1ql']}
[2019-12-10 23:16:49,432] - [basetestcase:400] INFO - --- End of cluster statistics ---
[2019-12-10 23:16:49,432] - [basetestcase:462] INFO - ==============  basetestcase cleanup was started for test #5 test_multi_create_query_explain_drop_index ==============
[2019-12-10 23:16:49,495] - [bucket_helper:143] INFO - deleting existing buckets [u'default'] on 127.0.0.1
[2019-12-10 23:16:49,496] - [bucket_helper:145] INFO - remove bucket default ...
[2019-12-10 23:16:50,775] - [bucket_helper:159] INFO - deleted bucket : default from 127.0.0.1
[2019-12-10 23:16:50,775] - [bucket_helper:237] INFO - waiting for bucket deletion to complete....
[2019-12-10 23:16:50,781] - [rest_client:133] INFO - node 127.0.0.1 existing buckets : []
[2019-12-10 23:16:50,782] - [bucket_helper:169] INFO - sleep 2 seconds to make sure all buckets ([u'default']) were deleted completely.
[2019-12-10 23:16:52,852] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9000
[2019-12-10 23:16:52,859] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9000 is running
[2019-12-10 23:16:52,866] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9001
[2019-12-10 23:16:52,872] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9001 is running
[2019-12-10 23:16:52,878] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9002
[2019-12-10 23:16:52,885] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9002 is running
[2019-12-10 23:16:52,892] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9003
[2019-12-10 23:16:52,898] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9003 is running
[2019-12-10 23:16:52,899] - [basetestcase:480] INFO - ==============  basetestcase cleanup was finished for test #5 test_multi_create_query_explain_drop_index ==============
ok

----------------------------------------------------------------------
Ran 1 test in 35.914s

OK
test_multi_create_query_explain_drop_index (2i.indexscans_2i.SecondaryIndexingScanTests) ... Cluster instance shutdown with force
summary so far suite 2i.indexscans_2i.SecondaryIndexingScanTests , pass 5 , fail 0
testrunner logs, diags and results are available under /opt/build/testrunner/logs/testrunner-19-Dec-10_23-14-02/test_5
Logs will be stored at /opt/build/testrunner/logs/testrunner-19-Dec-10_23-14-02/test_6

./testrunner -i b/resources/dev-4-nodes-xdcr_n1ql_2i.ini -p makefile=True -t 2i.indexscans_2i.SecondaryIndexingScanTests.test_multi_create_query_explain_drop_index,groups=simple:equals:no_orderby_groupby:range,dataset=default,doc-per-day=1,use_gsi_for_primary=True,use_gsi_for_secondary=True,doc_ops=True,update_ops_per=.5,run_async=True,scan_consistency=request_plus,GROUP=gsi

Test Input params:
{'doc-per-day': '1', 'use_gsi_for_primary': 'True', 'GROUP': 'gsi', 'use_gsi_for_secondary': 'True', 'gsi_type': 'forestdb', 'doc_ops': 'True', 'makefile': 'True', 'conf_file': 'conf/simple_2i_n1ql.conf', 'dataset': 'default', 'cluster_name': 'dev-4-nodes-xdcr_n1ql_2i', 'num_nodes': 4, 'update_ops_per': '.5', 'ini': 'b/resources/dev-4-nodes-xdcr_n1ql_2i.ini', 'groups': 'simple:equals:no_orderby_groupby:range', 'case_number': 6, 'scan_consistency': 'request_plus', 'run_async': 'True', 'logs_folder': '/opt/build/testrunner/logs/testrunner-19-Dec-10_23-14-02/test_6', 'spec': 'simple_2i_n1ql'}
[2019-12-10 23:16:52,986] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:16:52,986] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:16:53,508] - [rest_client:2127] INFO - Node version in cluster 6.5.0-4928-rel-enterprise
[2019-12-10 23:16:53,522] - [rest_client:2137] INFO - Node versions in cluster [u'6.5.0-4928-rel-enterprise']
[2019-12-10 23:16:53,522] - [basetestcase:218] INFO - ==============  basetestcase setup was started for test #6 test_multi_create_query_explain_drop_index==============
[2019-12-10 23:16:53,614] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9000
[2019-12-10 23:16:53,620] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9000 is running
[2019-12-10 23:16:53,627] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9001
[2019-12-10 23:16:53,633] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9001 is running
[2019-12-10 23:16:53,639] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9002
[2019-12-10 23:16:53,645] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9002 is running
[2019-12-10 23:16:53,651] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9003
[2019-12-10 23:16:53,657] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9003 is running
[2019-12-10 23:16:53,658] - [basetestcase:265] INFO - initializing cluster
[2019-12-10 23:16:53,996] - [task:132] INFO - server: ip:127.0.0.1 port:9000 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_0@cb.local', 'uptime': u'165', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9000', 'storage': [], 'moxi': 11211, 'port': u'9000', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12000, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'index', u'kv', u'n1ql'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13574291456, 'memoryTotal': 15466930176, 'memoryQuota': 7650, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:16:54,004] - [task:171] INFO - quota for index service will be 256 MB
[2019-12-10 23:16:54,004] - [task:173] INFO - set index quota to node 127.0.0.1 
[2019-12-10 23:16:54,005] - [rest_client:965] INFO - pools/default params : indexMemoryQuota=256
[2019-12-10 23:16:54,016] - [rest_client:953] INFO - pools/default params : memoryQuota=7650
[2019-12-10 23:16:54,034] - [rest_client:931] INFO - /node/controller/setupServices params on 127.0.0.1:9000:services=kv%2Cindex%2Cn1ql&password=asdasd&user=Administrator&hostname=127.0.0.1%3A9000
[2019-12-10 23:16:54,050] - [rest_client:840] ERROR - POST http://127.0.0.1:9000//node/controller/setupServices body: services=kv%2Cindex%2Cn1ql&password=asdasd&user=Administrator&hostname=127.0.0.1%3A9000 headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown ["cannot change node services after cluster is provisioned"] auth: Administrator:asdasd
[2019-12-10 23:16:54,050] - [rest_client:936] INFO - This node is already provisioned with services, we do not consider this as failure for test case
[2019-12-10 23:16:54,051] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9000:username=Administrator&password=asdasd&port=9000
[2019-12-10 23:16:54,111] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:16:54,111] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:16:54,586] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9000/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:16:54,667] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:16:54,675] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9000: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:16:54,675] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:16:54,681] - [rest_client:840] ERROR - POST http://127.0.0.1:9000/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:16:54,708] - [task:132] INFO - server: ip:127.0.0.1 port:9001 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_1@cb.local', 'uptime': u'164', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9001', 'storage': [], 'moxi': 11211, 'port': u'9001', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12002, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13580251136, 'memoryTotal': 15466930176, 'memoryQuota': 7906, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:16:54,719] - [rest_client:953] INFO - pools/default params : memoryQuota=7906
[2019-12-10 23:16:54,727] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9001:username=Administrator&password=asdasd&port=9001
[2019-12-10 23:16:54,787] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:16:54,787] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:16:55,277] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9001/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:16:55,362] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:16:55,370] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9001: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:16:55,371] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:16:55,376] - [rest_client:840] ERROR - POST http://127.0.0.1:9001/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:16:55,403] - [task:132] INFO - server: ip:127.0.0.1 port:9002 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_2@cb.local', 'uptime': u'169', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9002', 'storage': [], 'moxi': 11211, 'port': u'9002', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12004, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13630156800, 'memoryTotal': 15466930176, 'memoryQuota': 7906, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:16:55,413] - [rest_client:953] INFO - pools/default params : memoryQuota=7906
[2019-12-10 23:16:55,420] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9002:username=Administrator&password=asdasd&port=9002
[2019-12-10 23:16:55,476] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:16:55,476] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:16:55,969] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9002/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:16:56,055] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:16:56,063] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9002: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:16:56,064] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:16:56,069] - [rest_client:840] ERROR - POST http://127.0.0.1:9002/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:16:56,096] - [task:132] INFO - server: ip:127.0.0.1 port:9003 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_3@cb.local', 'uptime': u'169', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9003', 'storage': [], 'moxi': 11211, 'port': u'9003', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12006, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13630529536, 'memoryTotal': 15466930176, 'memoryQuota': 7906, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:16:56,106] - [rest_client:953] INFO - pools/default params : memoryQuota=7906
[2019-12-10 23:16:56,115] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9003:username=Administrator&password=asdasd&port=9003
[2019-12-10 23:16:56,172] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:16:56,172] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:16:56,647] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9003/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:16:56,731] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:16:56,738] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9003: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:16:56,739] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:16:56,744] - [rest_client:840] ERROR - POST http://127.0.0.1:9003/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:16:56,775] - [basetestcase:2218] INFO - **** add built-in 'cbadminbucket' user to node 127.0.0.1 ****
[2019-12-10 23:16:56,882] - [basetestcase:530] INFO - sleep for 5 secs.  ...
[2019-12-10 23:17:01,885] - [basetestcase:2223] INFO - **** add 'admin' role to 'cbadminbucket' user ****
[2019-12-10 23:17:01,910] - [basetestcase:291] INFO - done initializing cluster
[2019-12-10 23:17:01,915] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:17:01,915] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:17:02,414] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9000/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:17:02,502] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:17:02,504] - [basetestcase:2770] INFO - Enabled diag/eval for non-local hosts from 127.0.0.1
[2019-12-10 23:17:02,800] - [rest_client:2430] INFO - http://127.0.0.1:9000/pools/default/buckets with param: bucketType=membase&evictionPolicy=valueOnly&threadsNumber=3&ramQuotaMB=7650&authType=sasl&name=default&flushEnabled=1&replicaNumber=1&compressionMode=passive&saslPassword=None&replicaIndex=1
[2019-12-10 23:17:02,825] - [rest_client:2455] INFO - 0.02 seconds to create bucket default
[2019-12-10 23:17:02,826] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:17:02,963] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:17:03,040] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:17:03,084] - [task:351] WARNING - vbucket map not ready after try 0
[2019-12-10 23:17:03,084] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:17:03,168] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:17:03,248] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:17:03,289] - [task:351] WARNING - vbucket map not ready after try 1
[2019-12-10 23:17:03,290] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:17:03,365] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:17:03,524] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:17:03,566] - [task:351] WARNING - vbucket map not ready after try 2
[2019-12-10 23:17:03,566] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:17:03,641] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:17:03,755] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:17:03,796] - [task:351] WARNING - vbucket map not ready after try 3
[2019-12-10 23:17:03,797] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:17:03,874] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:17:04,026] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:17:04,068] - [task:351] WARNING - vbucket map not ready after try 4
[2019-12-10 23:17:04,069] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:17:04,150] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:17:04,264] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:17:04,305] - [task:351] WARNING - vbucket map not ready after try 5
[2019-12-10 23:17:04,358] - [basetestcase:383] INFO - ==============  basetestcase setup was finished for test #6 test_multi_create_query_explain_drop_index ==============
[2019-12-10 23:17:04,381] - [basetestcase:397] INFO - ------- Cluster statistics -------
[2019-12-10 23:17:04,382] - [basetestcase:399] INFO - 127.0.0.1:9000 => {'swap_mem_used': 0, 'cpu_utilization': 17.3469387755102, 'mem_free': 13635612672, 'swap_mem_total': 0, 'mem_total': 15466930176, 'services': [u'index', u'kv', u'n1ql']}
[2019-12-10 23:17:04,382] - [basetestcase:400] INFO - --- End of cluster statistics ---
[2019-12-10 23:17:04,394] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is warmup
[2019-12-10 23:17:04,395] - [newtuq:22] INFO - Waiting for cluster to become healthy
[2019-12-10 23:17:04,395] - [basetestcase:530] INFO - sleep for 5 secs.  ...
[2019-12-10 23:17:09,404] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:17:09,417] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is healthy
[2019-12-10 23:17:09,418] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:17:09,431] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is healthy
[2019-12-10 23:17:09,431] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:17:09,444] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is healthy
[2019-12-10 23:17:09,444] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:17:09,456] - [basetestcase:2436] INFO - list of index nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:17:09,475] - [rest_client:1706] INFO - {'indexer.settings.storage_mode': 'forestdb'} set
[2019-12-10 23:17:09,509] - [rest_client:1706] INFO - {'indexer.api.enableTestServer': True} set
[2019-12-10 23:17:09,513] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:17:09,513] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:17:11,167] - [basetestcase:2545] INFO - create 2016 to default documents...
[2019-12-10 23:17:11,217] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:17:13,538] - [basetestcase:2557] INFO - LOAD IS FINISHED
[2019-12-10 23:17:13,563] - [newtuq:73] INFO - {'update': {'start': 0, 'end': 0}, 'remaining': {'start': 0, 'end': 1}}
[2019-12-10 23:17:14,616] - [basetestcase:2436] INFO - list of n1ql nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:17:14,616] - [newtuq:83] INFO - ip:127.0.0.1 port:9000 ssh_username:Administrator
[2019-12-10 23:17:14,617] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2019-12-10 23:17:14,623] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2019-12-10 23:17:14,708] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 81.620656ms
[2019-12-10 23:17:14,708] - [tuq_helper:147] INFO - RUN QUERY CREATE PRIMARY INDEX ON default  USING GSI
[2019-12-10 23:17:14,718] - [rest_client:3465] INFO - query params : statement=CREATE+PRIMARY+INDEX+ON+default++USING+GSI
[2019-12-10 23:17:15,467] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 743.984703ms
[2019-12-10 23:17:15,468] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2019-12-10 23:17:15,478] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2019-12-10 23:17:15,497] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 14.70604ms
[2019-12-10 23:17:15,527] - [basetestcase:2436] INFO - list of index nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:17:16,515] - [tuq_helper:147] INFO - RUN QUERY CREATE INDEX employeea40f2abd7c474e889afc57f299156309job_title ON default(job_title) USING GSI  WITH {'nodes': ['127.0.0.1:9000'], 'defer_build': True}
[2019-12-10 23:17:16,523] - [rest_client:3465] INFO - query params : statement=CREATE+INDEX+employeea40f2abd7c474e889afc57f299156309job_title+ON+default%28job_title%29+USING+GSI++WITH+%7B%27nodes%27%3A+%5B%27127.0.0.1%3A9000%27%5D%2C+%27defer_build%27%3A+True%7D
[2019-12-10 23:17:16,771] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 245.005089ms
[2019-12-10 23:17:16,771] - [base_2i:109] INFO - BUILD INDEX on default(employeea40f2abd7c474e889afc57f299156309job_title) USING GSI
[2019-12-10 23:17:17,772] - [tuq_helper:147] INFO - RUN QUERY BUILD INDEX on default(employeea40f2abd7c474e889afc57f299156309job_title) USING GSI
[2019-12-10 23:17:17,779] - [rest_client:3465] INFO - query params : statement=BUILD+INDEX+on+default%28employeea40f2abd7c474e889afc57f299156309job_title%29+USING+GSI
[2019-12-10 23:17:17,847] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 65.138739ms
[2019-12-10 23:17:18,849] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employeea40f2abd7c474e889afc57f299156309job_title'
[2019-12-10 23:17:18,857] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employeea40f2abd7c474e889afc57f299156309job_title%27
[2019-12-10 23:17:18,873] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 13.361459ms
[2019-12-10 23:17:19,874] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employeea40f2abd7c474e889afc57f299156309job_title'
[2019-12-10 23:17:19,881] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employeea40f2abd7c474e889afc57f299156309job_title%27
[2019-12-10 23:17:19,901] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 16.889121ms
[2019-12-10 23:17:20,515] - [basetestcase:2545] INFO - update 0 to default documents...
[2019-12-10 23:17:20,570] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:17:21,408] - [basetestcase:2557] INFO - LOAD IS FINISHED
[2019-12-10 23:17:21,904] - [task:2607] INFO -  <<<<< START Executing Query EXPLAIN SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2019-12-10 23:17:21,904] - [tuq_helper:147] INFO - RUN QUERY EXPLAIN SELECT * FROM default WHERE  job_title = "Sales" 
[2019-12-10 23:17:21,914] - [rest_client:3465] INFO - query params : statement=EXPLAIN+SELECT+%2A+FROM+default+WHERE++job_title+%3D+%22Sales%22+
[2019-12-10 23:17:21,921] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 4.439295ms
[2019-12-10 23:17:21,921] - [task:2616] INFO - {u'status': u'success', u'metrics': {u'elapsedTime': u'4.439295ms', u'executionTime': u'4.322494ms', u'resultSize': 727, u'resultCount': 1}, u'results': [{u'text': u'SELECT * FROM default WHERE  job_title = "Sales"', u'plan': {u'#operator': u'Sequence', u'~children': [{u'index': u'employeea40f2abd7c474e889afc57f299156309job_title', u'namespace': u'default', u'keyspace': u'default', u'#operator': u'IndexScan3', u'index_projection': {u'primary_key': True}, u'spans': [{u'range': [{u'high': u'"Sales"', u'low': u'"Sales"', u'inclusion': 3}], u'exact': True}], u'using': u'gsi', u'index_id': u'4a0d4cb055dacb67'}, {u'keyspace': u'default', u'#operator': u'Fetch', u'namespace': u'default'}, {u'#operator': u'Parallel', u'~child': {u'#operator': u'Sequence', u'~children': [{u'#operator': u'Filter', u'condition': u'((`default`.`job_title`) = "Sales")'}, {u'#operator': u'InitialProject', u'result_terms': [{u'expr': u'self', u'star': True}]}, {u'#operator': u'FinalProject'}]}}]}}], u'requestID': u'81f6bf4f-6ef3-40f3-af4b-a22766fb00b4', u'signature': u'json'}
[2019-12-10 23:17:21,922] - [task:2617] INFO -  <<<<< Done Executing Query EXPLAIN SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2019-12-10 23:17:21,922] - [task:2645] INFO -  <<<<< Done VERIFYING Query EXPLAIN SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2019-12-10 23:17:21,923] - [base_2i:314] INFO - Query : SELECT * FROM default WHERE  job_title = "Sales" 
[2019-12-10 23:17:21,923] - [tuq_generators:70] INFO - FROM clause ===== is default
[2019-12-10 23:17:21,923] - [tuq_generators:72] INFO - WHERE clause ===== is   doc["job_title"] == "Sales" 
[2019-12-10 23:17:21,924] - [tuq_generators:74] INFO - UNNEST clause ===== is None
[2019-12-10 23:17:21,924] - [tuq_generators:76] INFO - SELECT clause ===== is {"*" : doc,}
[2019-12-10 23:17:22,924] - [task:2607] INFO -  <<<<< START Executing Query SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2019-12-10 23:17:22,924] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM default WHERE  job_title = "Sales" 
[2019-12-10 23:17:22,931] - [rest_client:3465] INFO - query params : scan_consistency=request_plus&statement=SELECT+%2A+FROM+default+WHERE++job_title+%3D+%22Sales%22+
[2019-12-10 23:17:23,081] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 127.005348ms
[2019-12-10 23:17:23,103] - [tuq_helper:214] INFO -  Analyzing Actual Result
[2019-12-10 23:17:23,104] - [tuq_helper:216] INFO -  Analyzing Expected Result
[2019-12-10 23:17:23,107] - [task:2617] INFO -  <<<<< Done Executing Query SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2019-12-10 23:17:23,107] - [task:2645] INFO -  <<<<< Done VERIFYING Query SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2019-12-10 23:17:24,108] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employeea40f2abd7c474e889afc57f299156309job_title'
[2019-12-10 23:17:24,115] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employeea40f2abd7c474e889afc57f299156309job_title%27
[2019-12-10 23:17:24,130] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 11.774602ms
[2019-12-10 23:17:24,130] - [tuq_helper:147] INFO - RUN QUERY DROP INDEX default.employeea40f2abd7c474e889afc57f299156309job_title USING GSI
[2019-12-10 23:17:24,137] - [rest_client:3465] INFO - query params : statement=DROP+INDEX+default.employeea40f2abd7c474e889afc57f299156309job_title+USING+GSI
[2019-12-10 23:17:24,201] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 60.187036ms
[2019-12-10 23:17:24,202] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employeea40f2abd7c474e889afc57f299156309job_title'
[2019-12-10 23:17:24,215] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employeea40f2abd7c474e889afc57f299156309job_title%27
[2019-12-10 23:17:24,228] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 10.084386ms
[2019-12-10 23:17:24,263] - [basetestcase:2436] INFO - list of index nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:17:24,268] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:17:24,268] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:17:24,909] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9000: True content: "/opt/build/ns_server/logs/n_0" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))).
[2019-12-10 23:17:24,910] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: zgrep "panic" "/opt/build/ns_server/logs/n_0"/indexer.log* | wc -l
[2019-12-10 23:17:25,006] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:17:25,023] - [basetestcase:2436] INFO - list of kv nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:17:25,028] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:17:25,028] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:17:25,699] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9000: True content: "/opt/build/ns_server/logs/n_0" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))).
[2019-12-10 23:17:25,700] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: zgrep "panic" "/opt/build/ns_server/logs/n_0"/projector.log* | wc -l
[2019-12-10 23:17:25,810] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:17:25,832] - [basetestcase:2436] INFO - list of n1ql nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:17:25,833] - [tuq_helper:516] INFO - CHECK FOR PRIMARY INDEXES
[2019-12-10 23:17:25,833] - [tuq_helper:523] INFO - DROP PRIMARY INDEX ON default USING GSI
[2019-12-10 23:17:25,833] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2019-12-10 23:17:25,841] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2019-12-10 23:17:25,852] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 8.488723ms
[2019-12-10 23:17:25,853] - [tuq_helper:147] INFO - RUN QUERY DROP PRIMARY INDEX ON default USING GSI
[2019-12-10 23:17:25,860] - [rest_client:3465] INFO - query params : statement=DROP+PRIMARY+INDEX+ON+default+USING+GSI
[2019-12-10 23:17:25,908] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 44.506657ms
[2019-12-10 23:17:25,908] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: rm -f /opt/couchbase/var/lib/couchbase/meta;rm -f /tmp/log_upr_client.sock
[2019-12-10 23:17:26,010] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:17:26,011] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: killall cbq-engine
[2019-12-10 23:17:26,104] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:17:26,105] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: killall tuqtng
[2019-12-10 23:17:26,216] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:17:26,217] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: killall indexer
[2019-12-10 23:17:26,337] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:17:26,367] - [basetestcase:397] INFO - ------- Cluster statistics -------
[2019-12-10 23:17:26,368] - [basetestcase:399] INFO - 127.0.0.1:9000 => {'swap_mem_used': 0, 'cpu_utilization': 18.41359773371105, 'mem_free': 13495848960, 'swap_mem_total': 0, 'mem_total': 15466930176, 'services': [u'index', u'kv', u'n1ql']}
[2019-12-10 23:17:26,368] - [basetestcase:400] INFO - --- End of cluster statistics ---
[2019-12-10 23:17:26,368] - [basetestcase:462] INFO - ==============  basetestcase cleanup was started for test #6 test_multi_create_query_explain_drop_index ==============
[2019-12-10 23:17:26,435] - [bucket_helper:143] INFO - deleting existing buckets [u'default'] on 127.0.0.1
[2019-12-10 23:17:26,435] - [bucket_helper:145] INFO - remove bucket default ...
[2019-12-10 23:17:27,119] - [bucket_helper:159] INFO - deleted bucket : default from 127.0.0.1
[2019-12-10 23:17:27,119] - [bucket_helper:237] INFO - waiting for bucket deletion to complete....
[2019-12-10 23:17:27,125] - [rest_client:133] INFO - node 127.0.0.1 existing buckets : []
[2019-12-10 23:17:27,125] - [bucket_helper:169] INFO - sleep 2 seconds to make sure all buckets ([u'default']) were deleted completely.
[2019-12-10 23:17:29,195] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9000
[2019-12-10 23:17:29,201] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9000 is running
[2019-12-10 23:17:29,207] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9001
[2019-12-10 23:17:29,214] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9001 is running
[2019-12-10 23:17:29,221] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9002
[2019-12-10 23:17:29,227] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9002 is running
[2019-12-10 23:17:29,233] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9003
[2019-12-10 23:17:29,240] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9003 is running
[2019-12-10 23:17:29,240] - [basetestcase:480] INFO - ==============  basetestcase cleanup was finished for test #6 test_multi_create_query_explain_drop_index ==============
ok

----------------------------------------------------------------------
Ran 1 test in 36.274s

OK
test_multi_create_query_explain_drop_index (2i.indexscans_2i.SecondaryIndexingScanTests) ... Cluster instance shutdown with force
summary so far suite 2i.indexscans_2i.SecondaryIndexingScanTests , pass 6 , fail 0
testrunner logs, diags and results are available under /opt/build/testrunner/logs/testrunner-19-Dec-10_23-14-02/test_6
Logs will be stored at /opt/build/testrunner/logs/testrunner-19-Dec-10_23-14-02/test_7

./testrunner -i b/resources/dev-4-nodes-xdcr_n1ql_2i.ini -p makefile=True -t 2i.indexscans_2i.SecondaryIndexingScanTests.test_multi_create_query_explain_drop_index,groups=simple:equals:no_orderby_groupby:range,dataset=default,doc-per-day=1,use_gsi_for_primary=True,use_gsi_for_secondary=True,doc_ops=True,expiry_ops_per=.5,run_async=True,scan_consistency=request_plus,GROUP=gsi

Test Input params:
{'doc-per-day': '1', 'use_gsi_for_primary': 'True', 'GROUP': 'gsi', 'use_gsi_for_secondary': 'True', 'gsi_type': 'forestdb', 'doc_ops': 'True', 'case_number': 7, 'makefile': 'True', 'conf_file': 'conf/simple_2i_n1ql.conf', 'dataset': 'default', 'cluster_name': 'dev-4-nodes-xdcr_n1ql_2i', 'expiry_ops_per': '.5', 'ini': 'b/resources/dev-4-nodes-xdcr_n1ql_2i.ini', 'groups': 'simple:equals:no_orderby_groupby:range', 'num_nodes': 4, 'scan_consistency': 'request_plus', 'run_async': 'True', 'logs_folder': '/opt/build/testrunner/logs/testrunner-19-Dec-10_23-14-02/test_7', 'spec': 'simple_2i_n1ql'}
[2019-12-10 23:17:29,327] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:17:29,327] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:17:29,919] - [rest_client:2127] INFO - Node version in cluster 6.5.0-4928-rel-enterprise
[2019-12-10 23:17:29,934] - [rest_client:2137] INFO - Node versions in cluster [u'6.5.0-4928-rel-enterprise']
[2019-12-10 23:17:29,934] - [basetestcase:218] INFO - ==============  basetestcase setup was started for test #7 test_multi_create_query_explain_drop_index==============
[2019-12-10 23:17:30,026] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9000
[2019-12-10 23:17:30,032] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9000 is running
[2019-12-10 23:17:30,038] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9001
[2019-12-10 23:17:30,044] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9001 is running
[2019-12-10 23:17:30,051] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9002
[2019-12-10 23:17:30,058] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9002 is running
[2019-12-10 23:17:30,065] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9003
[2019-12-10 23:17:30,071] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9003 is running
[2019-12-10 23:17:30,072] - [basetestcase:265] INFO - initializing cluster
[2019-12-10 23:17:30,337] - [task:132] INFO - server: ip:127.0.0.1 port:9000 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_0@cb.local', 'uptime': u'204', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9000', 'storage': [], 'moxi': 11211, 'port': u'9000', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12000, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'index', u'kv', u'n1ql'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13522538496, 'memoryTotal': 15466930176, 'memoryQuota': 7650, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:17:30,345] - [task:171] INFO - quota for index service will be 256 MB
[2019-12-10 23:17:30,345] - [task:173] INFO - set index quota to node 127.0.0.1 
[2019-12-10 23:17:30,346] - [rest_client:965] INFO - pools/default params : indexMemoryQuota=256
[2019-12-10 23:17:30,360] - [rest_client:953] INFO - pools/default params : memoryQuota=7650
[2019-12-10 23:17:30,387] - [rest_client:931] INFO - /node/controller/setupServices params on 127.0.0.1:9000:services=kv%2Cindex%2Cn1ql&password=asdasd&user=Administrator&hostname=127.0.0.1%3A9000
[2019-12-10 23:17:30,394] - [rest_client:840] ERROR - POST http://127.0.0.1:9000//node/controller/setupServices body: services=kv%2Cindex%2Cn1ql&password=asdasd&user=Administrator&hostname=127.0.0.1%3A9000 headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown ["cannot change node services after cluster is provisioned"] auth: Administrator:asdasd
[2019-12-10 23:17:30,394] - [rest_client:936] INFO - This node is already provisioned with services, we do not consider this as failure for test case
[2019-12-10 23:17:30,395] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9000:username=Administrator&password=asdasd&port=9000
[2019-12-10 23:17:30,465] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:17:30,465] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:17:31,032] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9000/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:17:31,124] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:17:31,133] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9000: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:17:31,134] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:17:31,139] - [rest_client:840] ERROR - POST http://127.0.0.1:9000/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:17:31,166] - [task:132] INFO - server: ip:127.0.0.1 port:9001 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_1@cb.local', 'uptime': u'204', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9001', 'storage': [], 'moxi': 11211, 'port': u'9001', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12002, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13591207936, 'memoryTotal': 15466930176, 'memoryQuota': 7906, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:17:31,177] - [rest_client:953] INFO - pools/default params : memoryQuota=7906
[2019-12-10 23:17:31,184] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9001:username=Administrator&password=asdasd&port=9001
[2019-12-10 23:17:31,240] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:17:31,240] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:17:31,792] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9001/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:17:31,886] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:17:31,895] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9001: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:17:31,895] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:17:31,901] - [rest_client:840] ERROR - POST http://127.0.0.1:9001/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:17:31,928] - [task:132] INFO - server: ip:127.0.0.1 port:9002 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_2@cb.local', 'uptime': u'204', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9002', 'storage': [], 'moxi': 11211, 'port': u'9002', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12004, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13590011904, 'memoryTotal': 15466930176, 'memoryQuota': 7906, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:17:31,938] - [rest_client:953] INFO - pools/default params : memoryQuota=7906
[2019-12-10 23:17:31,945] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9002:username=Administrator&password=asdasd&port=9002
[2019-12-10 23:17:32,003] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:17:32,003] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:17:32,560] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9002/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:17:32,651] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:17:32,659] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9002: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:17:32,660] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:17:32,665] - [rest_client:840] ERROR - POST http://127.0.0.1:9002/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:17:32,692] - [task:132] INFO - server: ip:127.0.0.1 port:9003 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_3@cb.local', 'uptime': u'204', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9003', 'storage': [], 'moxi': 11211, 'port': u'9003', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12006, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13591588864, 'memoryTotal': 15466930176, 'memoryQuota': 7906, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:17:32,702] - [rest_client:953] INFO - pools/default params : memoryQuota=7906
[2019-12-10 23:17:32,709] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9003:username=Administrator&password=asdasd&port=9003
[2019-12-10 23:17:32,768] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:17:32,768] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:17:33,326] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9003/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:17:33,419] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:17:33,426] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9003: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:17:33,427] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:17:33,433] - [rest_client:840] ERROR - POST http://127.0.0.1:9003/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:17:33,463] - [basetestcase:2218] INFO - **** add built-in 'cbadminbucket' user to node 127.0.0.1 ****
[2019-12-10 23:17:33,571] - [basetestcase:530] INFO - sleep for 5 secs.  ...
[2019-12-10 23:17:38,577] - [basetestcase:2223] INFO - **** add 'admin' role to 'cbadminbucket' user ****
[2019-12-10 23:17:38,599] - [basetestcase:291] INFO - done initializing cluster
[2019-12-10 23:17:38,605] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:17:38,605] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:17:39,169] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9000/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:17:39,269] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:17:39,270] - [basetestcase:2770] INFO - Enabled diag/eval for non-local hosts from 127.0.0.1
[2019-12-10 23:17:39,486] - [rest_client:2430] INFO - http://127.0.0.1:9000/pools/default/buckets with param: bucketType=membase&evictionPolicy=valueOnly&threadsNumber=3&ramQuotaMB=7650&authType=sasl&name=default&flushEnabled=1&replicaNumber=1&compressionMode=passive&saslPassword=None&replicaIndex=1
[2019-12-10 23:17:39,504] - [rest_client:2455] INFO - 0.02 seconds to create bucket default
[2019-12-10 23:17:39,505] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:17:39,644] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:17:39,726] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:17:39,770] - [task:351] WARNING - vbucket map not ready after try 0
[2019-12-10 23:17:39,770] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:17:39,861] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:17:39,943] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:17:39,983] - [task:351] WARNING - vbucket map not ready after try 1
[2019-12-10 23:17:39,984] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:17:40,061] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:17:40,240] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:17:40,280] - [task:351] WARNING - vbucket map not ready after try 2
[2019-12-10 23:17:40,281] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:17:40,356] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:17:40,479] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:17:40,525] - [task:351] WARNING - vbucket map not ready after try 3
[2019-12-10 23:17:40,525] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:17:40,619] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:17:40,773] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:17:40,815] - [task:351] WARNING - vbucket map not ready after try 4
[2019-12-10 23:17:40,815] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:17:40,891] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:17:41,019] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:17:41,060] - [task:351] WARNING - vbucket map not ready after try 5
[2019-12-10 23:17:41,094] - [basetestcase:383] INFO - ==============  basetestcase setup was finished for test #7 test_multi_create_query_explain_drop_index ==============
[2019-12-10 23:17:41,117] - [basetestcase:397] INFO - ------- Cluster statistics -------
[2019-12-10 23:17:41,118] - [basetestcase:399] INFO - 127.0.0.1:9000 => {'swap_mem_used': 0, 'cpu_utilization': 22.90076335877863, 'mem_free': 13595250688, 'swap_mem_total': 0, 'mem_total': 15466930176, 'services': [u'index', u'kv', u'n1ql']}
[2019-12-10 23:17:41,118] - [basetestcase:400] INFO - --- End of cluster statistics ---
[2019-12-10 23:17:41,130] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is warmup
[2019-12-10 23:17:41,130] - [newtuq:22] INFO - Waiting for cluster to become healthy
[2019-12-10 23:17:41,130] - [basetestcase:530] INFO - sleep for 5 secs.  ...
[2019-12-10 23:17:46,144] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:17:46,158] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is healthy
[2019-12-10 23:17:46,158] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:17:46,172] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is healthy
[2019-12-10 23:17:46,172] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:17:46,186] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is healthy
[2019-12-10 23:17:46,186] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:17:46,198] - [basetestcase:2436] INFO - list of index nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:17:46,218] - [rest_client:1706] INFO - {'indexer.settings.storage_mode': 'forestdb'} set
[2019-12-10 23:17:46,250] - [rest_client:1706] INFO - {'indexer.api.enableTestServer': True} set
[2019-12-10 23:17:46,253] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:17:46,253] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:17:48,046] - [basetestcase:2545] INFO - create 2016 to default documents...
[2019-12-10 23:17:48,097] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:17:50,309] - [basetestcase:2557] INFO - LOAD IS FINISHED
[2019-12-10 23:17:50,332] - [newtuq:73] INFO - {'remaining': {'start': 0, 'end': 1}, 'expiry': {'start': 0, 'end': 0}}
[2019-12-10 23:17:51,196] - [basetestcase:2436] INFO - list of n1ql nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:17:51,196] - [newtuq:83] INFO - ip:127.0.0.1 port:9000 ssh_username:Administrator
[2019-12-10 23:17:51,196] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2019-12-10 23:17:51,203] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2019-12-10 23:17:51,296] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 89.881977ms
[2019-12-10 23:17:51,296] - [tuq_helper:147] INFO - RUN QUERY CREATE PRIMARY INDEX ON default  USING GSI
[2019-12-10 23:17:51,306] - [rest_client:3465] INFO - query params : statement=CREATE+PRIMARY+INDEX+ON+default++USING+GSI
[2019-12-10 23:17:52,004] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 693.515121ms
[2019-12-10 23:17:52,004] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2019-12-10 23:17:52,014] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2019-12-10 23:17:52,028] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 10.208901ms
[2019-12-10 23:17:52,057] - [basetestcase:2436] INFO - list of index nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:17:53,046] - [tuq_helper:147] INFO - RUN QUERY CREATE INDEX employeeec025b1d9f1740489e80868b9443d194job_title ON default(job_title) USING GSI  WITH {'nodes': ['127.0.0.1:9000'], 'defer_build': True}
[2019-12-10 23:17:53,054] - [rest_client:3465] INFO - query params : statement=CREATE+INDEX+employeeec025b1d9f1740489e80868b9443d194job_title+ON+default%28job_title%29+USING+GSI++WITH+%7B%27nodes%27%3A+%5B%27127.0.0.1%3A9000%27%5D%2C+%27defer_build%27%3A+True%7D
[2019-12-10 23:17:53,307] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 251.167096ms
[2019-12-10 23:17:53,308] - [base_2i:109] INFO - BUILD INDEX on default(employeeec025b1d9f1740489e80868b9443d194job_title) USING GSI
[2019-12-10 23:17:54,308] - [tuq_helper:147] INFO - RUN QUERY BUILD INDEX on default(employeeec025b1d9f1740489e80868b9443d194job_title) USING GSI
[2019-12-10 23:17:54,316] - [rest_client:3465] INFO - query params : statement=BUILD+INDEX+on+default%28employeeec025b1d9f1740489e80868b9443d194job_title%29+USING+GSI
[2019-12-10 23:17:54,383] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 63.723591ms
[2019-12-10 23:17:55,384] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employeeec025b1d9f1740489e80868b9443d194job_title'
[2019-12-10 23:17:55,391] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employeeec025b1d9f1740489e80868b9443d194job_title%27
[2019-12-10 23:17:55,408] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 14.0797ms
[2019-12-10 23:17:56,409] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employeeec025b1d9f1740489e80868b9443d194job_title'
[2019-12-10 23:17:56,416] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employeeec025b1d9f1740489e80868b9443d194job_title%27
[2019-12-10 23:17:56,430] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 11.053271ms
[2019-12-10 23:17:57,020] - [basetestcase:2545] INFO - update 0 to default documents...
[2019-12-10 23:17:57,184] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:17:58,263] - [basetestcase:2557] INFO - LOAD IS FINISHED
[2019-12-10 23:17:58,337] - [data_helper:301] INFO - dict:{'username': 'Administrator', 'ip': '127.0.0.1', 'password': 'asdasd', 'port': u'9000'}
[2019-12-10 23:17:58,338] - [data_helper:302] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:17:58,389] - [cluster_helper:371] INFO - Setting flush param on server {'username': 'Administrator', 'ip': '127.0.0.1', 'password': 'asdasd', 'port': u'9000'}, exp_pager_stime to 10 on default
[2019-12-10 23:17:58,389] - [mc_bin_client:648] INFO - setting param: exp_pager_stime 10
[2019-12-10 23:17:58,390] - [cluster_helper:385] INFO - Setting flush param on server {'username': 'Administrator', 'ip': '127.0.0.1', 'password': 'asdasd', 'port': u'9000'}, exp_pager_stime to 10, result: (2621814255, 0, '')
[2019-12-10 23:17:58,431] - [task:2607] INFO -  <<<<< START Executing Query EXPLAIN SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2019-12-10 23:17:58,432] - [tuq_helper:147] INFO - RUN QUERY EXPLAIN SELECT * FROM default WHERE  job_title = "Sales" 
[2019-12-10 23:17:58,439] - [rest_client:3465] INFO - query params : statement=EXPLAIN+SELECT+%2A+FROM+default+WHERE++job_title+%3D+%22Sales%22+
[2019-12-10 23:17:58,445] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 3.280397ms
[2019-12-10 23:17:58,445] - [task:2616] INFO - {u'status': u'success', u'metrics': {u'elapsedTime': u'3.280397ms', u'executionTime': u'3.234283ms', u'resultSize': 727, u'resultCount': 1}, u'results': [{u'text': u'SELECT * FROM default WHERE  job_title = "Sales"', u'plan': {u'#operator': u'Sequence', u'~children': [{u'index': u'employeeec025b1d9f1740489e80868b9443d194job_title', u'namespace': u'default', u'keyspace': u'default', u'#operator': u'IndexScan3', u'index_projection': {u'primary_key': True}, u'spans': [{u'range': [{u'high': u'"Sales"', u'low': u'"Sales"', u'inclusion': 3}], u'exact': True}], u'using': u'gsi', u'index_id': u'caf6eaaa1cc93238'}, {u'keyspace': u'default', u'#operator': u'Fetch', u'namespace': u'default'}, {u'#operator': u'Parallel', u'~child': {u'#operator': u'Sequence', u'~children': [{u'#operator': u'Filter', u'condition': u'((`default`.`job_title`) = "Sales")'}, {u'#operator': u'InitialProject', u'result_terms': [{u'expr': u'self', u'star': True}]}, {u'#operator': u'FinalProject'}]}}]}}], u'requestID': u'dc434d37-ad94-4e39-9c3b-1de9467224b3', u'signature': u'json'}
[2019-12-10 23:17:58,445] - [task:2617] INFO -  <<<<< Done Executing Query EXPLAIN SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2019-12-10 23:17:58,446] - [task:2645] INFO -  <<<<< Done VERIFYING Query EXPLAIN SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2019-12-10 23:17:58,446] - [base_2i:314] INFO - Query : SELECT * FROM default WHERE  job_title = "Sales" 
[2019-12-10 23:17:58,447] - [tuq_generators:70] INFO - FROM clause ===== is default
[2019-12-10 23:17:58,447] - [tuq_generators:72] INFO - WHERE clause ===== is   doc["job_title"] == "Sales" 
[2019-12-10 23:17:58,447] - [tuq_generators:74] INFO - UNNEST clause ===== is None
[2019-12-10 23:17:58,448] - [tuq_generators:76] INFO - SELECT clause ===== is {"*" : doc,}
[2019-12-10 23:17:59,447] - [task:2607] INFO -  <<<<< START Executing Query SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2019-12-10 23:17:59,447] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM default WHERE  job_title = "Sales" 
[2019-12-10 23:17:59,455] - [rest_client:3465] INFO - query params : scan_consistency=request_plus&statement=SELECT+%2A+FROM+default+WHERE++job_title+%3D+%22Sales%22+
[2019-12-10 23:17:59,612] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 137.40368ms
[2019-12-10 23:17:59,633] - [tuq_helper:214] INFO -  Analyzing Actual Result
[2019-12-10 23:17:59,633] - [tuq_helper:216] INFO -  Analyzing Expected Result
[2019-12-10 23:17:59,637] - [task:2617] INFO -  <<<<< Done Executing Query SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2019-12-10 23:17:59,637] - [task:2645] INFO -  <<<<< Done VERIFYING Query SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2019-12-10 23:18:00,638] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employeeec025b1d9f1740489e80868b9443d194job_title'
[2019-12-10 23:18:00,645] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employeeec025b1d9f1740489e80868b9443d194job_title%27
[2019-12-10 23:18:00,658] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 9.589929ms
[2019-12-10 23:18:00,658] - [tuq_helper:147] INFO - RUN QUERY DROP INDEX default.employeeec025b1d9f1740489e80868b9443d194job_title USING GSI
[2019-12-10 23:18:00,665] - [rest_client:3465] INFO - query params : statement=DROP+INDEX+default.employeeec025b1d9f1740489e80868b9443d194job_title+USING+GSI
[2019-12-10 23:18:00,728] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 59.248514ms
[2019-12-10 23:18:00,728] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employeeec025b1d9f1740489e80868b9443d194job_title'
[2019-12-10 23:18:00,736] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employeeec025b1d9f1740489e80868b9443d194job_title%27
[2019-12-10 23:18:00,751] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 12.047817ms
[2019-12-10 23:18:00,786] - [basetestcase:2436] INFO - list of index nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:18:00,791] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:18:00,791] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:18:01,538] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9000: True content: "/opt/build/ns_server/logs/n_0" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))).
[2019-12-10 23:18:01,539] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: zgrep "panic" "/opt/build/ns_server/logs/n_0"/indexer.log* | wc -l
[2019-12-10 23:18:01,651] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:18:01,667] - [basetestcase:2436] INFO - list of kv nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:18:01,673] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:18:01,673] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:18:02,418] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9000: True content: "/opt/build/ns_server/logs/n_0" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))).
[2019-12-10 23:18:02,418] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: zgrep "panic" "/opt/build/ns_server/logs/n_0"/projector.log* | wc -l
[2019-12-10 23:18:02,536] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:18:02,557] - [basetestcase:2436] INFO - list of n1ql nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:18:02,557] - [tuq_helper:516] INFO - CHECK FOR PRIMARY INDEXES
[2019-12-10 23:18:02,557] - [tuq_helper:523] INFO - DROP PRIMARY INDEX ON default USING GSI
[2019-12-10 23:18:02,558] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2019-12-10 23:18:02,565] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2019-12-10 23:18:02,579] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 11.356052ms
[2019-12-10 23:18:02,580] - [tuq_helper:147] INFO - RUN QUERY DROP PRIMARY INDEX ON default USING GSI
[2019-12-10 23:18:02,587] - [rest_client:3465] INFO - query params : statement=DROP+PRIMARY+INDEX+ON+default+USING+GSI
[2019-12-10 23:18:02,646] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 55.975916ms
[2019-12-10 23:18:02,647] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: rm -f /opt/couchbase/var/lib/couchbase/meta;rm -f /tmp/log_upr_client.sock
[2019-12-10 23:18:02,763] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:18:02,764] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: killall cbq-engine
[2019-12-10 23:18:02,866] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:18:02,868] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: killall tuqtng
[2019-12-10 23:18:02,995] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:18:02,996] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: killall indexer
[2019-12-10 23:18:03,122] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:18:03,158] - [basetestcase:397] INFO - ------- Cluster statistics -------
[2019-12-10 23:18:03,159] - [basetestcase:399] INFO - 127.0.0.1:9000 => {'swap_mem_used': 0, 'cpu_utilization': 5.428571428571429, 'mem_free': 13436657664, 'swap_mem_total': 0, 'mem_total': 15466930176, 'services': [u'index', u'kv', u'n1ql']}
[2019-12-10 23:18:03,159] - [basetestcase:400] INFO - --- End of cluster statistics ---
[2019-12-10 23:18:03,159] - [basetestcase:462] INFO - ==============  basetestcase cleanup was started for test #7 test_multi_create_query_explain_drop_index ==============
[2019-12-10 23:18:03,235] - [bucket_helper:143] INFO - deleting existing buckets [u'default'] on 127.0.0.1
[2019-12-10 23:18:03,236] - [bucket_helper:145] INFO - remove bucket default ...
[2019-12-10 23:18:03,828] - [bucket_helper:159] INFO - deleted bucket : default from 127.0.0.1
[2019-12-10 23:18:03,828] - [bucket_helper:237] INFO - waiting for bucket deletion to complete....
[2019-12-10 23:18:03,834] - [rest_client:133] INFO - node 127.0.0.1 existing buckets : []
[2019-12-10 23:18:03,834] - [bucket_helper:169] INFO - sleep 2 seconds to make sure all buckets ([u'default']) were deleted completely.
[2019-12-10 23:18:05,904] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9000
[2019-12-10 23:18:05,910] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9000 is running
[2019-12-10 23:18:05,917] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9001
[2019-12-10 23:18:05,923] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9001 is running
[2019-12-10 23:18:05,932] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9002
[2019-12-10 23:18:05,938] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9002 is running
[2019-12-10 23:18:05,945] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9003
[2019-12-10 23:18:05,952] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9003 is running
[2019-12-10 23:18:05,952] - [basetestcase:480] INFO - ==============  basetestcase cleanup was finished for test #7 test_multi_create_query_explain_drop_index ==============
ok

----------------------------------------------------------------------
Ran 1 test in 36.645s

OK
test_multi_create_query_explain_drop_index (2i.indexscans_2i.SecondaryIndexingScanTests) ... Cluster instance shutdown with force
summary so far suite 2i.indexscans_2i.SecondaryIndexingScanTests , pass 7 , fail 0
testrunner logs, diags and results are available under /opt/build/testrunner/logs/testrunner-19-Dec-10_23-14-02/test_7
Logs will be stored at /opt/build/testrunner/logs/testrunner-19-Dec-10_23-14-02/test_8

./testrunner -i b/resources/dev-4-nodes-xdcr_n1ql_2i.ini -p makefile=True -t 2i.indexscans_2i.SecondaryIndexingScanTests.test_multi_create_query_explain_drop_index,groups=simple:equals:no_orderby_groupby:range,dataset=default,doc-per-day=1,use_gsi_for_primary=True,use_gsi_for_secondary=True,doc_ops=True,create_ops_per=.5,run_async=True,scan_consistency=request_plus,GROUP=gsi

Test Input params:
{'doc-per-day': '1', 'use_gsi_for_primary': 'True', 'GROUP': 'gsi', 'use_gsi_for_secondary': 'True', 'gsi_type': 'forestdb', 'doc_ops': 'True', 'makefile': 'True', 'conf_file': 'conf/simple_2i_n1ql.conf', 'dataset': 'default', 'cluster_name': 'dev-4-nodes-xdcr_n1ql_2i', 'num_nodes': 4, 'run_async': 'True', 'groups': 'simple:equals:no_orderby_groupby:range', 'case_number': 8, 'scan_consistency': 'request_plus', 'create_ops_per': '.5', 'logs_folder': '/opt/build/testrunner/logs/testrunner-19-Dec-10_23-14-02/test_8', 'spec': 'simple_2i_n1ql', 'ini': 'b/resources/dev-4-nodes-xdcr_n1ql_2i.ini'}
[2019-12-10 23:18:06,051] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:18:06,052] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:18:06,753] - [rest_client:2127] INFO - Node version in cluster 6.5.0-4928-rel-enterprise
[2019-12-10 23:18:06,768] - [rest_client:2137] INFO - Node versions in cluster [u'6.5.0-4928-rel-enterprise']
[2019-12-10 23:18:06,768] - [basetestcase:218] INFO - ==============  basetestcase setup was started for test #8 test_multi_create_query_explain_drop_index==============
[2019-12-10 23:18:06,863] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9000
[2019-12-10 23:18:06,869] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9000 is running
[2019-12-10 23:18:06,876] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9001
[2019-12-10 23:18:06,882] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9001 is running
[2019-12-10 23:18:06,890] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9002
[2019-12-10 23:18:06,897] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9002 is running
[2019-12-10 23:18:06,904] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9003
[2019-12-10 23:18:06,910] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9003 is running
[2019-12-10 23:18:06,911] - [basetestcase:265] INFO - initializing cluster
[2019-12-10 23:18:07,061] - [task:132] INFO - server: ip:127.0.0.1 port:9000 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_0@cb.local', 'uptime': u'239', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9000', 'storage': [], 'moxi': 11211, 'port': u'9000', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12000, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'index', u'kv', u'n1ql'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13477457920, 'memoryTotal': 15466930176, 'memoryQuota': 7650, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:18:07,069] - [task:171] INFO - quota for index service will be 256 MB
[2019-12-10 23:18:07,069] - [task:173] INFO - set index quota to node 127.0.0.1 
[2019-12-10 23:18:07,069] - [rest_client:965] INFO - pools/default params : indexMemoryQuota=256
[2019-12-10 23:18:07,078] - [rest_client:953] INFO - pools/default params : memoryQuota=7650
[2019-12-10 23:18:07,096] - [rest_client:931] INFO - /node/controller/setupServices params on 127.0.0.1:9000:services=kv%2Cindex%2Cn1ql&password=asdasd&user=Administrator&hostname=127.0.0.1%3A9000
[2019-12-10 23:18:07,104] - [rest_client:840] ERROR - POST http://127.0.0.1:9000//node/controller/setupServices body: services=kv%2Cindex%2Cn1ql&password=asdasd&user=Administrator&hostname=127.0.0.1%3A9000 headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown ["cannot change node services after cluster is provisioned"] auth: Administrator:asdasd
[2019-12-10 23:18:07,104] - [rest_client:936] INFO - This node is already provisioned with services, we do not consider this as failure for test case
[2019-12-10 23:18:07,104] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9000:username=Administrator&password=asdasd&port=9000
[2019-12-10 23:18:07,169] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:18:07,169] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:18:07,838] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9000/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:18:07,953] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:18:07,966] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9000: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:18:07,967] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:18:07,972] - [rest_client:840] ERROR - POST http://127.0.0.1:9000/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:18:08,000] - [task:132] INFO - server: ip:127.0.0.1 port:9001 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_1@cb.local', 'uptime': u'239', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9001', 'storage': [], 'moxi': 11211, 'port': u'9001', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12002, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13477670912, 'memoryTotal': 15466930176, 'memoryQuota': 7906, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:18:08,010] - [rest_client:953] INFO - pools/default params : memoryQuota=7906
[2019-12-10 23:18:08,019] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9001:username=Administrator&password=asdasd&port=9001
[2019-12-10 23:18:08,078] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:18:08,079] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:18:08,726] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9001/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:18:08,834] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:18:08,841] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9001: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:18:08,842] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:18:08,847] - [rest_client:840] ERROR - POST http://127.0.0.1:9001/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:18:08,874] - [task:132] INFO - server: ip:127.0.0.1 port:9002 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_2@cb.local', 'uptime': u'239', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9002', 'storage': [], 'moxi': 11211, 'port': u'9002', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12004, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13478445056, 'memoryTotal': 15466930176, 'memoryQuota': 7906, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:18:08,884] - [rest_client:953] INFO - pools/default params : memoryQuota=7906
[2019-12-10 23:18:08,893] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9002:username=Administrator&password=asdasd&port=9002
[2019-12-10 23:18:08,952] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:18:08,952] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:18:09,620] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9002/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:18:09,728] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:18:09,737] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9002: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:18:09,738] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:18:09,745] - [rest_client:840] ERROR - POST http://127.0.0.1:9002/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:18:09,773] - [task:132] INFO - server: ip:127.0.0.1 port:9003 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_3@cb.local', 'uptime': u'239', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9003', 'storage': [], 'moxi': 11211, 'port': u'9003', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12006, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13478445056, 'memoryTotal': 15466930176, 'memoryQuota': 7906, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:18:09,784] - [rest_client:953] INFO - pools/default params : memoryQuota=7906
[2019-12-10 23:18:09,799] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9003:username=Administrator&password=asdasd&port=9003
[2019-12-10 23:18:09,861] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:18:09,861] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:18:10,538] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9003/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:18:10,642] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:18:10,650] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9003: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:18:10,651] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:18:10,656] - [rest_client:840] ERROR - POST http://127.0.0.1:9003/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:18:10,691] - [basetestcase:2218] INFO - **** add built-in 'cbadminbucket' user to node 127.0.0.1 ****
[2019-12-10 23:18:10,801] - [basetestcase:530] INFO - sleep for 5 secs.  ...
[2019-12-10 23:18:15,804] - [basetestcase:2223] INFO - **** add 'admin' role to 'cbadminbucket' user ****
[2019-12-10 23:18:15,828] - [basetestcase:291] INFO - done initializing cluster
[2019-12-10 23:18:15,833] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:18:15,833] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:18:16,473] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9000/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:18:16,581] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:18:16,582] - [basetestcase:2770] INFO - Enabled diag/eval for non-local hosts from 127.0.0.1
[2019-12-10 23:18:16,711] - [rest_client:2430] INFO - http://127.0.0.1:9000/pools/default/buckets with param: bucketType=membase&evictionPolicy=valueOnly&threadsNumber=3&ramQuotaMB=7650&authType=sasl&name=default&flushEnabled=1&replicaNumber=1&compressionMode=passive&saslPassword=None&replicaIndex=1
[2019-12-10 23:18:16,731] - [rest_client:2455] INFO - 0.02 seconds to create bucket default
[2019-12-10 23:18:16,732] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:18:16,880] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:18:16,966] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:18:17,007] - [task:351] WARNING - vbucket map not ready after try 0
[2019-12-10 23:18:17,008] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:18:17,093] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:18:17,168] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:18:17,208] - [task:351] WARNING - vbucket map not ready after try 1
[2019-12-10 23:18:17,209] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:18:17,283] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:18:17,476] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:18:17,526] - [task:351] WARNING - vbucket map not ready after try 2
[2019-12-10 23:18:17,527] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:18:17,602] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:18:17,742] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:18:17,786] - [task:351] WARNING - vbucket map not ready after try 3
[2019-12-10 23:18:17,787] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:18:17,897] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:18:17,981] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:18:18,082] - [task:351] WARNING - vbucket map not ready after try 4
[2019-12-10 23:18:18,083] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:18:18,156] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:18:18,232] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:18:18,273] - [task:351] WARNING - vbucket map not ready after try 5
[2019-12-10 23:18:18,280] - [basetestcase:383] INFO - ==============  basetestcase setup was finished for test #8 test_multi_create_query_explain_drop_index ==============
[2019-12-10 23:18:18,304] - [basetestcase:397] INFO - ------- Cluster statistics -------
[2019-12-10 23:18:18,304] - [basetestcase:399] INFO - 127.0.0.1:9000 => {'swap_mem_used': 0, 'cpu_utilization': 20.41343669250646, 'mem_free': 13534961664, 'swap_mem_total': 0, 'mem_total': 15466930176, 'services': [u'index', u'kv', u'n1ql']}
[2019-12-10 23:18:18,304] - [basetestcase:400] INFO - --- End of cluster statistics ---
[2019-12-10 23:18:18,316] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is warmup
[2019-12-10 23:18:18,316] - [newtuq:22] INFO - Waiting for cluster to become healthy
[2019-12-10 23:18:18,316] - [basetestcase:530] INFO - sleep for 5 secs.  ...
[2019-12-10 23:18:23,330] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:18:23,344] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is healthy
[2019-12-10 23:18:23,344] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:18:23,357] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is healthy
[2019-12-10 23:18:23,358] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:18:23,371] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is healthy
[2019-12-10 23:18:23,371] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:18:23,384] - [basetestcase:2436] INFO - list of index nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:18:23,408] - [rest_client:1706] INFO - {'indexer.settings.storage_mode': 'forestdb'} set
[2019-12-10 23:18:23,439] - [rest_client:1706] INFO - {'indexer.api.enableTestServer': True} set
[2019-12-10 23:18:23,443] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:18:23,443] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:18:25,394] - [basetestcase:2545] INFO - create 2016 to default documents...
[2019-12-10 23:18:25,446] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:18:27,534] - [basetestcase:2557] INFO - LOAD IS FINISHED
[2019-12-10 23:18:27,558] - [newtuq:73] INFO - {'create': {'start': 1, 'end': 2}, 'remaining': {'start': 0, 'end': 1}}
[2019-12-10 23:18:28,925] - [basetestcase:2436] INFO - list of n1ql nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:18:28,925] - [newtuq:83] INFO - ip:127.0.0.1 port:9000 ssh_username:Administrator
[2019-12-10 23:18:28,926] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2019-12-10 23:18:28,932] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2019-12-10 23:18:29,021] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 85.897449ms
[2019-12-10 23:18:29,021] - [tuq_helper:147] INFO - RUN QUERY CREATE PRIMARY INDEX ON default  USING GSI
[2019-12-10 23:18:29,028] - [rest_client:3465] INFO - query params : statement=CREATE+PRIMARY+INDEX+ON+default++USING+GSI
[2019-12-10 23:18:29,745] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 713.711026ms
[2019-12-10 23:18:29,745] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2019-12-10 23:18:29,753] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2019-12-10 23:18:29,768] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 11.383346ms
[2019-12-10 23:18:29,803] - [basetestcase:2436] INFO - list of index nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:18:30,790] - [tuq_helper:147] INFO - RUN QUERY CREATE INDEX employee51ef943b446d46d4a21a4e726888c12cjob_title ON default(job_title) USING GSI  WITH {'nodes': ['127.0.0.1:9000'], 'defer_build': True}
[2019-12-10 23:18:30,798] - [rest_client:3465] INFO - query params : statement=CREATE+INDEX+employee51ef943b446d46d4a21a4e726888c12cjob_title+ON+default%28job_title%29+USING+GSI++WITH+%7B%27nodes%27%3A+%5B%27127.0.0.1%3A9000%27%5D%2C+%27defer_build%27%3A+True%7D
[2019-12-10 23:18:31,059] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 258.225154ms
[2019-12-10 23:18:31,060] - [base_2i:109] INFO - BUILD INDEX on default(employee51ef943b446d46d4a21a4e726888c12cjob_title) USING GSI
[2019-12-10 23:18:32,061] - [tuq_helper:147] INFO - RUN QUERY BUILD INDEX on default(employee51ef943b446d46d4a21a4e726888c12cjob_title) USING GSI
[2019-12-10 23:18:32,068] - [rest_client:3465] INFO - query params : statement=BUILD+INDEX+on+default%28employee51ef943b446d46d4a21a4e726888c12cjob_title%29+USING+GSI
[2019-12-10 23:18:32,135] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 64.157148ms
[2019-12-10 23:18:33,137] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee51ef943b446d46d4a21a4e726888c12cjob_title'
[2019-12-10 23:18:33,145] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee51ef943b446d46d4a21a4e726888c12cjob_title%27
[2019-12-10 23:18:33,162] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 14.448825ms
[2019-12-10 23:18:34,163] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee51ef943b446d46d4a21a4e726888c12cjob_title'
[2019-12-10 23:18:34,170] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee51ef943b446d46d4a21a4e726888c12cjob_title%27
[2019-12-10 23:18:34,186] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 13.616228ms
[2019-12-10 23:18:34,881] - [basetestcase:2545] INFO - create 2016 to default documents...
[2019-12-10 23:18:34,935] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:18:37,395] - [basetestcase:2557] INFO - LOAD IS FINISHED
[2019-12-10 23:18:38,191] - [task:2607] INFO -  <<<<< START Executing Query EXPLAIN SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2019-12-10 23:18:38,192] - [tuq_helper:147] INFO - RUN QUERY EXPLAIN SELECT * FROM default WHERE  job_title = "Sales" 
[2019-12-10 23:18:38,201] - [rest_client:3465] INFO - query params : statement=EXPLAIN+SELECT+%2A+FROM+default+WHERE++job_title+%3D+%22Sales%22+
[2019-12-10 23:18:38,208] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 4.040423ms
[2019-12-10 23:18:38,208] - [task:2616] INFO - {u'status': u'success', u'metrics': {u'elapsedTime': u'4.040423ms', u'executionTime': u'3.972799ms', u'resultSize': 727, u'resultCount': 1}, u'results': [{u'text': u'SELECT * FROM default WHERE  job_title = "Sales"', u'plan': {u'#operator': u'Sequence', u'~children': [{u'index': u'employee51ef943b446d46d4a21a4e726888c12cjob_title', u'namespace': u'default', u'keyspace': u'default', u'#operator': u'IndexScan3', u'index_projection': {u'primary_key': True}, u'spans': [{u'range': [{u'high': u'"Sales"', u'low': u'"Sales"', u'inclusion': 3}], u'exact': True}], u'using': u'gsi', u'index_id': u'e8fd8bc5fe9dcae4'}, {u'keyspace': u'default', u'#operator': u'Fetch', u'namespace': u'default'}, {u'#operator': u'Parallel', u'~child': {u'#operator': u'Sequence', u'~children': [{u'#operator': u'Filter', u'condition': u'((`default`.`job_title`) = "Sales")'}, {u'#operator': u'InitialProject', u'result_terms': [{u'expr': u'self', u'star': True}]}, {u'#operator': u'FinalProject'}]}}]}}], u'requestID': u'33d6f06c-bea1-484f-baa2-f45e827f5119', u'signature': u'json'}
[2019-12-10 23:18:38,208] - [task:2617] INFO -  <<<<< Done Executing Query EXPLAIN SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2019-12-10 23:18:38,209] - [task:2645] INFO -  <<<<< Done VERIFYING Query EXPLAIN SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2019-12-10 23:18:38,209] - [base_2i:314] INFO - Query : SELECT * FROM default WHERE  job_title = "Sales" 
[2019-12-10 23:18:38,210] - [tuq_generators:70] INFO - FROM clause ===== is default
[2019-12-10 23:18:38,210] - [tuq_generators:72] INFO - WHERE clause ===== is   doc["job_title"] == "Sales" 
[2019-12-10 23:18:38,211] - [tuq_generators:74] INFO - UNNEST clause ===== is None
[2019-12-10 23:18:38,211] - [tuq_generators:76] INFO - SELECT clause ===== is {"*" : doc,}
[2019-12-10 23:18:39,210] - [task:2607] INFO -  <<<<< START Executing Query SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2019-12-10 23:18:39,210] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM default WHERE  job_title = "Sales" 
[2019-12-10 23:18:39,218] - [rest_client:3465] INFO - query params : scan_consistency=request_plus&statement=SELECT+%2A+FROM+default+WHERE++job_title+%3D+%22Sales%22+
[2019-12-10 23:18:39,500] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 229.817008ms
[2019-12-10 23:18:39,541] - [tuq_helper:214] INFO -  Analyzing Actual Result
[2019-12-10 23:18:39,542] - [tuq_helper:216] INFO -  Analyzing Expected Result
[2019-12-10 23:18:39,551] - [task:2617] INFO -  <<<<< Done Executing Query SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2019-12-10 23:18:39,551] - [task:2645] INFO -  <<<<< Done VERIFYING Query SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2019-12-10 23:18:40,552] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee51ef943b446d46d4a21a4e726888c12cjob_title'
[2019-12-10 23:18:40,560] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee51ef943b446d46d4a21a4e726888c12cjob_title%27
[2019-12-10 23:18:40,575] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 11.888946ms
[2019-12-10 23:18:40,575] - [tuq_helper:147] INFO - RUN QUERY DROP INDEX default.employee51ef943b446d46d4a21a4e726888c12cjob_title USING GSI
[2019-12-10 23:18:40,582] - [rest_client:3465] INFO - query params : statement=DROP+INDEX+default.employee51ef943b446d46d4a21a4e726888c12cjob_title+USING+GSI
[2019-12-10 23:18:40,652] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 67.284616ms
[2019-12-10 23:18:40,652] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee51ef943b446d46d4a21a4e726888c12cjob_title'
[2019-12-10 23:18:40,662] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee51ef943b446d46d4a21a4e726888c12cjob_title%27
[2019-12-10 23:18:40,677] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 12.69593ms
[2019-12-10 23:18:40,712] - [basetestcase:2436] INFO - list of index nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:18:40,715] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:18:40,715] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:18:41,556] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9000: True content: "/opt/build/ns_server/logs/n_0" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))).
[2019-12-10 23:18:41,557] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: zgrep "panic" "/opt/build/ns_server/logs/n_0"/indexer.log* | wc -l
[2019-12-10 23:18:41,681] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:18:41,700] - [basetestcase:2436] INFO - list of kv nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:18:41,704] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:18:41,704] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:18:42,550] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9000: True content: "/opt/build/ns_server/logs/n_0" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))).
[2019-12-10 23:18:42,551] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: zgrep "panic" "/opt/build/ns_server/logs/n_0"/projector.log* | wc -l
[2019-12-10 23:18:42,675] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:18:42,692] - [basetestcase:2436] INFO - list of n1ql nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:18:42,692] - [tuq_helper:516] INFO - CHECK FOR PRIMARY INDEXES
[2019-12-10 23:18:42,692] - [tuq_helper:523] INFO - DROP PRIMARY INDEX ON default USING GSI
[2019-12-10 23:18:42,693] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2019-12-10 23:18:42,700] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2019-12-10 23:18:42,715] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 12.288847ms
[2019-12-10 23:18:42,715] - [tuq_helper:147] INFO - RUN QUERY DROP PRIMARY INDEX ON default USING GSI
[2019-12-10 23:18:42,724] - [rest_client:3465] INFO - query params : statement=DROP+PRIMARY+INDEX+ON+default+USING+GSI
[2019-12-10 23:18:42,785] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 58.386127ms
[2019-12-10 23:18:42,786] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: rm -f /opt/couchbase/var/lib/couchbase/meta;rm -f /tmp/log_upr_client.sock
[2019-12-10 23:18:42,918] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:18:42,919] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: killall cbq-engine
[2019-12-10 23:18:43,031] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:18:43,033] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: killall tuqtng
[2019-12-10 23:18:43,176] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:18:43,177] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: killall indexer
[2019-12-10 23:18:43,330] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:18:43,362] - [basetestcase:397] INFO - ------- Cluster statistics -------
[2019-12-10 23:18:43,363] - [basetestcase:399] INFO - 127.0.0.1:9000 => {'swap_mem_used': 0, 'cpu_utilization': 23.09859154929578, 'mem_free': 13363478528, 'swap_mem_total': 0, 'mem_total': 15466930176, 'services': [u'index', u'kv', u'n1ql']}
[2019-12-10 23:18:43,363] - [basetestcase:400] INFO - --- End of cluster statistics ---
[2019-12-10 23:18:43,364] - [basetestcase:462] INFO - ==============  basetestcase cleanup was started for test #8 test_multi_create_query_explain_drop_index ==============
[2019-12-10 23:18:43,748] - [bucket_helper:143] INFO - deleting existing buckets [u'default'] on 127.0.0.1
[2019-12-10 23:18:43,748] - [bucket_helper:145] INFO - remove bucket default ...
[2019-12-10 23:18:45,032] - [bucket_helper:159] INFO - deleted bucket : default from 127.0.0.1
[2019-12-10 23:18:45,032] - [bucket_helper:237] INFO - waiting for bucket deletion to complete....
[2019-12-10 23:18:45,038] - [rest_client:133] INFO - node 127.0.0.1 existing buckets : []
[2019-12-10 23:18:45,038] - [bucket_helper:169] INFO - sleep 2 seconds to make sure all buckets ([u'default']) were deleted completely.
[2019-12-10 23:18:47,106] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9000
[2019-12-10 23:18:47,113] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9000 is running
[2019-12-10 23:18:47,119] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9001
[2019-12-10 23:18:47,125] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9001 is running
[2019-12-10 23:18:47,132] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9002
[2019-12-10 23:18:47,138] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9002 is running
[2019-12-10 23:18:47,144] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9003
[2019-12-10 23:18:47,150] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9003 is running
[2019-12-10 23:18:47,151] - [basetestcase:480] INFO - ==============  basetestcase cleanup was finished for test #8 test_multi_create_query_explain_drop_index ==============
ok

----------------------------------------------------------------------
Ran 1 test in 41.118s

OK
test_multi_create_query_explain_drop_index (2i.indexscans_2i.SecondaryIndexingScanTests) ... Cluster instance shutdown with force
summary so far suite 2i.indexscans_2i.SecondaryIndexingScanTests , pass 8 , fail 0
testrunner logs, diags and results are available under /opt/build/testrunner/logs/testrunner-19-Dec-10_23-14-02/test_8
Logs will be stored at /opt/build/testrunner/logs/testrunner-19-Dec-10_23-14-02/test_9

./testrunner -i b/resources/dev-4-nodes-xdcr_n1ql_2i.ini -p makefile=True -t 2i.indexscans_2i.SecondaryIndexingScanTests.test_multi_create_query_explain_drop_index,groups=simple:equals:no_orderby_groupby:range,dataset=default,doc-per-day=1,use_gsi_for_primary=True,use_gsi_for_secondary=True,doc_ops=True,create_ops_per=.5,delete_ops_per=.2,update_ops_per=.2,run_async=True,scan_consistency=request_plus,GROUP=gsi

Test Input params:
{'doc-per-day': '1', 'use_gsi_for_primary': 'True', 'GROUP': 'gsi', 'use_gsi_for_secondary': 'True', 'gsi_type': 'forestdb', 'doc_ops': 'True', 'makefile': 'True', 'delete_ops_per': '.2', 'conf_file': 'conf/simple_2i_n1ql.conf', 'dataset': 'default', 'cluster_name': 'dev-4-nodes-xdcr_n1ql_2i', 'num_nodes': 4, 'update_ops_per': '.2', 'run_async': 'True', 'groups': 'simple:equals:no_orderby_groupby:range', 'case_number': 9, 'scan_consistency': 'request_plus', 'create_ops_per': '.5', 'logs_folder': '/opt/build/testrunner/logs/testrunner-19-Dec-10_23-14-02/test_9', 'spec': 'simple_2i_n1ql', 'ini': 'b/resources/dev-4-nodes-xdcr_n1ql_2i.ini'}
[2019-12-10 23:18:47,208] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:18:47,208] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:18:48,001] - [rest_client:2127] INFO - Node version in cluster 6.5.0-4928-rel-enterprise
[2019-12-10 23:18:48,016] - [rest_client:2137] INFO - Node versions in cluster [u'6.5.0-4928-rel-enterprise']
[2019-12-10 23:18:48,016] - [basetestcase:218] INFO - ==============  basetestcase setup was started for test #9 test_multi_create_query_explain_drop_index==============
[2019-12-10 23:18:48,108] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9000
[2019-12-10 23:18:48,115] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9000 is running
[2019-12-10 23:18:48,121] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9001
[2019-12-10 23:18:48,127] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9001 is running
[2019-12-10 23:18:48,134] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9002
[2019-12-10 23:18:48,140] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9002 is running
[2019-12-10 23:18:48,147] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9003
[2019-12-10 23:18:48,153] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9003 is running
[2019-12-10 23:18:48,154] - [basetestcase:265] INFO - initializing cluster
[2019-12-10 23:18:48,218] - [task:132] INFO - server: ip:127.0.0.1 port:9000 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_0@cb.local', 'uptime': u'279', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9000', 'storage': [], 'moxi': 11211, 'port': u'9000', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12000, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'index', u'kv', u'n1ql'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13356425216, 'memoryTotal': 15466930176, 'memoryQuota': 7650, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:18:48,226] - [task:171] INFO - quota for index service will be 256 MB
[2019-12-10 23:18:48,226] - [task:173] INFO - set index quota to node 127.0.0.1 
[2019-12-10 23:18:48,226] - [rest_client:965] INFO - pools/default params : indexMemoryQuota=256
[2019-12-10 23:18:48,237] - [rest_client:953] INFO - pools/default params : memoryQuota=7650
[2019-12-10 23:18:48,270] - [rest_client:931] INFO - /node/controller/setupServices params on 127.0.0.1:9000:services=kv%2Cindex%2Cn1ql&password=asdasd&user=Administrator&hostname=127.0.0.1%3A9000
[2019-12-10 23:18:48,277] - [rest_client:840] ERROR - POST http://127.0.0.1:9000//node/controller/setupServices body: services=kv%2Cindex%2Cn1ql&password=asdasd&user=Administrator&hostname=127.0.0.1%3A9000 headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown ["cannot change node services after cluster is provisioned"] auth: Administrator:asdasd
[2019-12-10 23:18:48,278] - [rest_client:936] INFO - This node is already provisioned with services, we do not consider this as failure for test case
[2019-12-10 23:18:48,278] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9000:username=Administrator&password=asdasd&port=9000
[2019-12-10 23:18:48,337] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:18:48,337] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:18:49,112] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9000/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:18:49,235] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:18:49,244] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9000: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:18:49,245] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:18:49,251] - [rest_client:840] ERROR - POST http://127.0.0.1:9000/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:18:49,278] - [task:132] INFO - server: ip:127.0.0.1 port:9001 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_1@cb.local', 'uptime': u'279', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9001', 'storage': [], 'moxi': 11211, 'port': u'9001', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12002, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13356068864, 'memoryTotal': 15466930176, 'memoryQuota': 7906, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:18:49,287] - [rest_client:953] INFO - pools/default params : memoryQuota=7906
[2019-12-10 23:18:49,296] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9001:username=Administrator&password=asdasd&port=9001
[2019-12-10 23:18:49,353] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:18:49,353] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:18:50,141] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9001/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:18:50,266] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:18:50,274] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9001: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:18:50,275] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:18:50,284] - [rest_client:840] ERROR - POST http://127.0.0.1:9001/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:18:50,312] - [task:132] INFO - server: ip:127.0.0.1 port:9002 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_2@cb.local', 'uptime': u'284', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9002', 'storage': [], 'moxi': 11211, 'port': u'9002', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12004, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13485473792, 'memoryTotal': 15466930176, 'memoryQuota': 7906, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:18:50,321] - [rest_client:953] INFO - pools/default params : memoryQuota=7906
[2019-12-10 23:18:50,330] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9002:username=Administrator&password=asdasd&port=9002
[2019-12-10 23:18:50,388] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:18:50,389] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:18:51,185] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9002/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:18:51,314] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:18:51,322] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9002: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:18:51,322] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:18:51,328] - [rest_client:840] ERROR - POST http://127.0.0.1:9002/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:18:51,355] - [task:132] INFO - server: ip:127.0.0.1 port:9003 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_3@cb.local', 'uptime': u'284', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9003', 'storage': [], 'moxi': 11211, 'port': u'9003', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12006, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13484023808, 'memoryTotal': 15466930176, 'memoryQuota': 7906, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:18:51,364] - [rest_client:953] INFO - pools/default params : memoryQuota=7906
[2019-12-10 23:18:51,372] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9003:username=Administrator&password=asdasd&port=9003
[2019-12-10 23:18:51,435] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:18:51,436] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:18:52,209] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9003/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:18:52,334] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:18:52,341] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9003: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:18:52,342] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:18:52,347] - [rest_client:840] ERROR - POST http://127.0.0.1:9003/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:18:52,378] - [basetestcase:2218] INFO - **** add built-in 'cbadminbucket' user to node 127.0.0.1 ****
[2019-12-10 23:18:52,492] - [basetestcase:530] INFO - sleep for 5 secs.  ...
[2019-12-10 23:18:57,496] - [basetestcase:2223] INFO - **** add 'admin' role to 'cbadminbucket' user ****
[2019-12-10 23:18:57,520] - [basetestcase:291] INFO - done initializing cluster
[2019-12-10 23:18:57,525] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:18:57,525] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:18:58,301] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9000/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:18:58,425] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:18:58,426] - [basetestcase:2770] INFO - Enabled diag/eval for non-local hosts from 127.0.0.1
[2019-12-10 23:18:59,419] - [rest_client:2430] INFO - http://127.0.0.1:9000/pools/default/buckets with param: bucketType=membase&evictionPolicy=valueOnly&threadsNumber=3&ramQuotaMB=7650&authType=sasl&name=default&flushEnabled=1&replicaNumber=1&compressionMode=passive&saslPassword=None&replicaIndex=1
[2019-12-10 23:18:59,439] - [rest_client:2455] INFO - 0.02 seconds to create bucket default
[2019-12-10 23:18:59,440] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:18:59,596] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:18:59,684] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:18:59,732] - [task:351] WARNING - vbucket map not ready after try 0
[2019-12-10 23:18:59,732] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:18:59,827] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:18:59,910] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:18:59,955] - [task:351] WARNING - vbucket map not ready after try 1
[2019-12-10 23:18:59,955] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:19:00,036] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:19:00,126] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:19:00,324] - [task:351] WARNING - vbucket map not ready after try 2
[2019-12-10 23:19:00,324] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:19:00,403] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:19:00,492] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:19:00,550] - [task:351] WARNING - vbucket map not ready after try 3
[2019-12-10 23:19:00,550] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:19:00,715] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:19:00,797] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:19:00,841] - [task:351] WARNING - vbucket map not ready after try 4
[2019-12-10 23:19:00,841] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:19:00,922] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:19:01,005] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:19:01,128] - [task:351] WARNING - vbucket map not ready after try 5
[2019-12-10 23:19:01,145] - [basetestcase:383] INFO - ==============  basetestcase setup was finished for test #9 test_multi_create_query_explain_drop_index ==============
[2019-12-10 23:19:01,169] - [basetestcase:397] INFO - ------- Cluster statistics -------
[2019-12-10 23:19:01,170] - [basetestcase:399] INFO - 127.0.0.1:9000 => {'swap_mem_used': 0, 'cpu_utilization': 8.354430379746836, 'mem_free': 13486297088, 'swap_mem_total': 0, 'mem_total': 15466930176, 'services': [u'index', u'kv', u'n1ql']}
[2019-12-10 23:19:01,170] - [basetestcase:400] INFO - --- End of cluster statistics ---
[2019-12-10 23:19:01,182] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is warmup
[2019-12-10 23:19:01,182] - [newtuq:22] INFO - Waiting for cluster to become healthy
[2019-12-10 23:19:01,183] - [basetestcase:530] INFO - sleep for 5 secs.  ...
[2019-12-10 23:19:06,195] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:19:06,209] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is healthy
[2019-12-10 23:19:06,209] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:19:06,223] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is healthy
[2019-12-10 23:19:06,224] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:19:06,237] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is healthy
[2019-12-10 23:19:06,237] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:19:06,251] - [basetestcase:2436] INFO - list of index nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:19:06,270] - [rest_client:1706] INFO - {'indexer.settings.storage_mode': 'forestdb'} set
[2019-12-10 23:19:06,308] - [rest_client:1706] INFO - {'indexer.api.enableTestServer': True} set
[2019-12-10 23:19:06,315] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:19:06,315] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:19:08,361] - [basetestcase:2545] INFO - create 2016 to default documents...
[2019-12-10 23:19:08,413] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:19:10,395] - [basetestcase:2557] INFO - LOAD IS FINISHED
[2019-12-10 23:19:10,416] - [newtuq:73] INFO - {'remaining': {'start': 0, 'end': 1}, 'create': {'start': 1, 'end': 2}, 'update': {'start': 0, 'end': 0}, 'delete': {'start': 0, 'end': 0}}
[2019-12-10 23:19:12,215] - [basetestcase:2436] INFO - list of n1ql nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:19:12,216] - [newtuq:83] INFO - ip:127.0.0.1 port:9000 ssh_username:Administrator
[2019-12-10 23:19:12,216] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2019-12-10 23:19:12,223] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2019-12-10 23:19:12,321] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 95.170445ms
[2019-12-10 23:19:12,322] - [tuq_helper:147] INFO - RUN QUERY CREATE PRIMARY INDEX ON default  USING GSI
[2019-12-10 23:19:12,329] - [rest_client:3465] INFO - query params : statement=CREATE+PRIMARY+INDEX+ON+default++USING+GSI
[2019-12-10 23:19:13,067] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 735.177217ms
[2019-12-10 23:19:13,067] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2019-12-10 23:19:13,076] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2019-12-10 23:19:13,092] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 13.522283ms
[2019-12-10 23:19:13,123] - [basetestcase:2436] INFO - list of index nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:19:14,112] - [tuq_helper:147] INFO - RUN QUERY CREATE INDEX employeefeccd273488942da98e1e813d00526a8job_title ON default(job_title) USING GSI  WITH {'nodes': ['127.0.0.1:9000'], 'defer_build': True}
[2019-12-10 23:19:14,119] - [rest_client:3465] INFO - query params : statement=CREATE+INDEX+employeefeccd273488942da98e1e813d00526a8job_title+ON+default%28job_title%29+USING+GSI++WITH+%7B%27nodes%27%3A+%5B%27127.0.0.1%3A9000%27%5D%2C+%27defer_build%27%3A+True%7D
[2019-12-10 23:19:14,377] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 255.483359ms
[2019-12-10 23:19:14,378] - [base_2i:109] INFO - BUILD INDEX on default(employeefeccd273488942da98e1e813d00526a8job_title) USING GSI
[2019-12-10 23:19:15,379] - [tuq_helper:147] INFO - RUN QUERY BUILD INDEX on default(employeefeccd273488942da98e1e813d00526a8job_title) USING GSI
[2019-12-10 23:19:15,387] - [rest_client:3465] INFO - query params : statement=BUILD+INDEX+on+default%28employeefeccd273488942da98e1e813d00526a8job_title%29+USING+GSI
[2019-12-10 23:19:15,461] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 70.647817ms
[2019-12-10 23:19:16,462] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employeefeccd273488942da98e1e813d00526a8job_title'
[2019-12-10 23:19:16,470] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employeefeccd273488942da98e1e813d00526a8job_title%27
[2019-12-10 23:19:16,488] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 14.972619ms
[2019-12-10 23:19:17,489] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employeefeccd273488942da98e1e813d00526a8job_title'
[2019-12-10 23:19:17,497] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employeefeccd273488942da98e1e813d00526a8job_title%27
[2019-12-10 23:19:17,514] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 14.767602ms
[2019-12-10 23:19:18,226] - [basetestcase:2545] INFO - create 2016 to default documents...
[2019-12-10 23:19:18,284] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:19:21,256] - [basetestcase:2557] INFO - LOAD IS FINISHED
[2019-12-10 23:19:22,215] - [basetestcase:2545] INFO - delete 0 to default documents...
[2019-12-10 23:19:22,273] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:19:23,340] - [basetestcase:2557] INFO - LOAD IS FINISHED
[2019-12-10 23:19:24,314] - [basetestcase:2545] INFO - update 0 to default documents...
[2019-12-10 23:19:24,372] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:19:25,455] - [basetestcase:2557] INFO - LOAD IS FINISHED
[2019-12-10 23:19:25,535] - [task:2607] INFO -  <<<<< START Executing Query EXPLAIN SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2019-12-10 23:19:25,536] - [tuq_helper:147] INFO - RUN QUERY EXPLAIN SELECT * FROM default WHERE  job_title = "Sales" 
[2019-12-10 23:19:25,546] - [rest_client:3465] INFO - query params : statement=EXPLAIN+SELECT+%2A+FROM+default+WHERE++job_title+%3D+%22Sales%22+
[2019-12-10 23:19:25,552] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 3.574445ms
[2019-12-10 23:19:25,552] - [task:2616] INFO - {u'status': u'success', u'metrics': {u'elapsedTime': u'3.574445ms', u'executionTime': u'3.460779ms', u'resultSize': 727, u'resultCount': 1}, u'results': [{u'text': u'SELECT * FROM default WHERE  job_title = "Sales"', u'plan': {u'#operator': u'Sequence', u'~children': [{u'index': u'employeefeccd273488942da98e1e813d00526a8job_title', u'namespace': u'default', u'keyspace': u'default', u'#operator': u'IndexScan3', u'index_projection': {u'primary_key': True}, u'spans': [{u'range': [{u'high': u'"Sales"', u'low': u'"Sales"', u'inclusion': 3}], u'exact': True}], u'using': u'gsi', u'index_id': u'87b5b5ed500bde24'}, {u'keyspace': u'default', u'#operator': u'Fetch', u'namespace': u'default'}, {u'#operator': u'Parallel', u'~child': {u'#operator': u'Sequence', u'~children': [{u'#operator': u'Filter', u'condition': u'((`default`.`job_title`) = "Sales")'}, {u'#operator': u'InitialProject', u'result_terms': [{u'expr': u'self', u'star': True}]}, {u'#operator': u'FinalProject'}]}}]}}], u'requestID': u'7d3694b5-5a72-4caf-a7f7-37163d4c5691', u'signature': u'json'}
[2019-12-10 23:19:25,552] - [task:2617] INFO -  <<<<< Done Executing Query EXPLAIN SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2019-12-10 23:19:25,553] - [task:2645] INFO -  <<<<< Done VERIFYING Query EXPLAIN SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2019-12-10 23:19:25,553] - [base_2i:314] INFO - Query : SELECT * FROM default WHERE  job_title = "Sales" 
[2019-12-10 23:19:25,554] - [tuq_generators:70] INFO - FROM clause ===== is default
[2019-12-10 23:19:25,554] - [tuq_generators:72] INFO - WHERE clause ===== is   doc["job_title"] == "Sales" 
[2019-12-10 23:19:25,554] - [tuq_generators:74] INFO - UNNEST clause ===== is None
[2019-12-10 23:19:25,555] - [tuq_generators:76] INFO - SELECT clause ===== is {"*" : doc,}
[2019-12-10 23:19:26,553] - [task:2607] INFO -  <<<<< START Executing Query SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2019-12-10 23:19:26,554] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM default WHERE  job_title = "Sales" 
[2019-12-10 23:19:26,561] - [rest_client:3465] INFO - query params : scan_consistency=request_plus&statement=SELECT+%2A+FROM+default+WHERE++job_title+%3D+%22Sales%22+
[2019-12-10 23:19:26,844] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 233.608441ms
[2019-12-10 23:19:26,886] - [tuq_helper:214] INFO -  Analyzing Actual Result
[2019-12-10 23:19:26,887] - [tuq_helper:216] INFO -  Analyzing Expected Result
[2019-12-10 23:19:26,896] - [task:2617] INFO -  <<<<< Done Executing Query SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2019-12-10 23:19:26,896] - [task:2645] INFO -  <<<<< Done VERIFYING Query SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2019-12-10 23:19:27,897] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employeefeccd273488942da98e1e813d00526a8job_title'
[2019-12-10 23:19:27,905] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employeefeccd273488942da98e1e813d00526a8job_title%27
[2019-12-10 23:19:27,920] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 12.940345ms
[2019-12-10 23:19:27,920] - [tuq_helper:147] INFO - RUN QUERY DROP INDEX default.employeefeccd273488942da98e1e813d00526a8job_title USING GSI
[2019-12-10 23:19:27,927] - [rest_client:3465] INFO - query params : statement=DROP+INDEX+default.employeefeccd273488942da98e1e813d00526a8job_title+USING+GSI
[2019-12-10 23:19:28,000] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 70.033954ms
[2019-12-10 23:19:28,000] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employeefeccd273488942da98e1e813d00526a8job_title'
[2019-12-10 23:19:28,009] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employeefeccd273488942da98e1e813d00526a8job_title%27
[2019-12-10 23:19:28,022] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 9.552001ms
[2019-12-10 23:19:28,054] - [basetestcase:2436] INFO - list of index nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:19:28,059] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:19:28,059] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:19:29,081] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9000: True content: "/opt/build/ns_server/logs/n_0" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))).
[2019-12-10 23:19:29,081] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: zgrep "panic" "/opt/build/ns_server/logs/n_0"/indexer.log* | wc -l
[2019-12-10 23:19:29,232] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:19:29,251] - [basetestcase:2436] INFO - list of kv nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:19:29,256] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:19:29,256] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:19:30,275] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9000: True content: "/opt/build/ns_server/logs/n_0" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))).
[2019-12-10 23:19:30,276] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: zgrep "panic" "/opt/build/ns_server/logs/n_0"/projector.log* | wc -l
[2019-12-10 23:19:30,431] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:19:30,449] - [basetestcase:2436] INFO - list of n1ql nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:19:30,449] - [tuq_helper:516] INFO - CHECK FOR PRIMARY INDEXES
[2019-12-10 23:19:30,449] - [tuq_helper:523] INFO - DROP PRIMARY INDEX ON default USING GSI
[2019-12-10 23:19:30,450] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2019-12-10 23:19:30,457] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2019-12-10 23:19:30,472] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 12.181466ms
[2019-12-10 23:19:30,472] - [tuq_helper:147] INFO - RUN QUERY DROP PRIMARY INDEX ON default USING GSI
[2019-12-10 23:19:30,479] - [rest_client:3465] INFO - query params : statement=DROP+PRIMARY+INDEX+ON+default+USING+GSI
[2019-12-10 23:19:30,543] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 60.908239ms
[2019-12-10 23:19:30,544] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: rm -f /opt/couchbase/var/lib/couchbase/meta;rm -f /tmp/log_upr_client.sock
[2019-12-10 23:19:30,705] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:19:30,706] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: killall cbq-engine
[2019-12-10 23:19:30,848] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:19:30,849] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: killall tuqtng
[2019-12-10 23:19:31,030] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:19:31,031] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: killall indexer
[2019-12-10 23:19:31,199] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:19:31,231] - [basetestcase:397] INFO - ------- Cluster statistics -------
[2019-12-10 23:19:31,231] - [basetestcase:399] INFO - 127.0.0.1:9000 => {'swap_mem_used': 0, 'cpu_utilization': 30.4225352112676, 'mem_free': 13308264448, 'swap_mem_total': 0, 'mem_total': 15466930176, 'services': [u'index', u'kv', u'n1ql']}
[2019-12-10 23:19:31,231] - [basetestcase:400] INFO - --- End of cluster statistics ---
[2019-12-10 23:19:31,232] - [basetestcase:462] INFO - ==============  basetestcase cleanup was started for test #9 test_multi_create_query_explain_drop_index ==============
[2019-12-10 23:19:31,286] - [bucket_helper:143] INFO - deleting existing buckets [u'default'] on 127.0.0.1
[2019-12-10 23:19:31,286] - [bucket_helper:145] INFO - remove bucket default ...
[2019-12-10 23:19:31,768] - [bucket_helper:159] INFO - deleted bucket : default from 127.0.0.1
[2019-12-10 23:19:31,769] - [bucket_helper:237] INFO - waiting for bucket deletion to complete....
[2019-12-10 23:19:31,774] - [rest_client:133] INFO - node 127.0.0.1 existing buckets : []
[2019-12-10 23:19:31,775] - [bucket_helper:169] INFO - sleep 2 seconds to make sure all buckets ([u'default']) were deleted completely.
[2019-12-10 23:19:33,846] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9000
[2019-12-10 23:19:33,853] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9000 is running
[2019-12-10 23:19:33,859] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9001
[2019-12-10 23:19:33,865] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9001 is running
[2019-12-10 23:19:33,872] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9002
[2019-12-10 23:19:33,878] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9002 is running
[2019-12-10 23:19:33,884] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9003
[2019-12-10 23:19:33,891] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9003 is running
[2019-12-10 23:19:33,892] - [basetestcase:480] INFO - ==============  basetestcase cleanup was finished for test #9 test_multi_create_query_explain_drop_index ==============
ok

----------------------------------------------------------------------
Ran 1 test in 46.702s

OK
test_multi_create_drop_index (2i.indexcreatedrop_2i.SecondaryIndexingCreateDropTests) ... Cluster instance shutdown with force
summary so far suite 2i.indexscans_2i.SecondaryIndexingScanTests , pass 9 , fail 0
testrunner logs, diags and results are available under /opt/build/testrunner/logs/testrunner-19-Dec-10_23-14-02/test_9
Logs will be stored at /opt/build/testrunner/logs/testrunner-19-Dec-10_23-14-02/test_10

./testrunner -i b/resources/dev-4-nodes-xdcr_n1ql_2i.ini -p makefile=True -t 2i.indexcreatedrop_2i.SecondaryIndexingCreateDropTests.test_multi_create_drop_index,groups=simple,dataset=default,doc-per-day=1,cbq_version=sherlock,skip_build_tuq=True,GROUP=views

Test Input params:
{'doc-per-day': '1', 'logs_folder': '/opt/build/testrunner/logs/testrunner-19-Dec-10_23-14-02/test_10', 'GROUP': 'views', 'gsi_type': 'forestdb', 'makefile': 'True', 'skip_build_tuq': 'True', 'conf_file': 'conf/simple_2i_n1ql.conf', 'dataset': 'default', 'cluster_name': 'dev-4-nodes-xdcr_n1ql_2i', 'ini': 'b/resources/dev-4-nodes-xdcr_n1ql_2i.ini', 'groups': 'simple', 'case_number': 10, 'num_nodes': 4, 'spec': 'simple_2i_n1ql', 'cbq_version': 'sherlock'}
[2019-12-10 23:19:34,033] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:19:34,033] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:19:34,965] - [rest_client:2127] INFO - Node version in cluster 6.5.0-4928-rel-enterprise
[2019-12-10 23:19:34,979] - [rest_client:2137] INFO - Node versions in cluster [u'6.5.0-4928-rel-enterprise']
[2019-12-10 23:19:34,979] - [basetestcase:218] INFO - ==============  basetestcase setup was started for test #10 test_multi_create_drop_index==============
[2019-12-10 23:19:35,074] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9000
[2019-12-10 23:19:35,080] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9000 is running
[2019-12-10 23:19:35,087] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9001
[2019-12-10 23:19:35,093] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9001 is running
[2019-12-10 23:19:35,099] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9002
[2019-12-10 23:19:35,106] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9002 is running
[2019-12-10 23:19:35,112] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9003
[2019-12-10 23:19:35,120] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9003 is running
[2019-12-10 23:19:35,121] - [basetestcase:265] INFO - initializing cluster
[2019-12-10 23:19:36,041] - [task:132] INFO - server: ip:127.0.0.1 port:9000 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_0@cb.local', 'uptime': u'329', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9000', 'storage': [], 'moxi': 11211, 'port': u'9000', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12000, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'index', u'kv', u'n1ql'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13340905472, 'memoryTotal': 15466930176, 'memoryQuota': 7650, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:19:36,051] - [task:171] INFO - quota for index service will be 256 MB
[2019-12-10 23:19:36,051] - [task:173] INFO - set index quota to node 127.0.0.1 
[2019-12-10 23:19:36,051] - [rest_client:965] INFO - pools/default params : indexMemoryQuota=256
[2019-12-10 23:19:36,061] - [rest_client:953] INFO - pools/default params : memoryQuota=7650
[2019-12-10 23:19:36,076] - [rest_client:931] INFO - /node/controller/setupServices params on 127.0.0.1:9000:services=kv%2Cindex%2Cn1ql&password=asdasd&user=Administrator&hostname=127.0.0.1%3A9000
[2019-12-10 23:19:36,095] - [rest_client:840] ERROR - POST http://127.0.0.1:9000//node/controller/setupServices body: services=kv%2Cindex%2Cn1ql&password=asdasd&user=Administrator&hostname=127.0.0.1%3A9000 headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown ["cannot change node services after cluster is provisioned"] auth: Administrator:asdasd
[2019-12-10 23:19:36,097] - [rest_client:936] INFO - This node is already provisioned with services, we do not consider this as failure for test case
[2019-12-10 23:19:36,097] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9000:username=Administrator&password=asdasd&port=9000
[2019-12-10 23:19:36,155] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:19:36,155] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:19:37,061] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9000/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:19:37,204] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:19:37,212] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9000: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:19:37,213] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:19:37,219] - [rest_client:840] ERROR - POST http://127.0.0.1:9000/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:19:37,245] - [task:132] INFO - server: ip:127.0.0.1 port:9001 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_1@cb.local', 'uptime': u'329', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9001', 'storage': [], 'moxi': 11211, 'port': u'9001', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12002, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13435203584, 'memoryTotal': 15466930176, 'memoryQuota': 7906, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:19:37,255] - [rest_client:953] INFO - pools/default params : memoryQuota=7906
[2019-12-10 23:19:37,264] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9001:username=Administrator&password=asdasd&port=9001
[2019-12-10 23:19:37,321] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:19:37,322] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:19:38,211] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9001/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:19:38,350] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:19:38,361] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9001: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:19:38,361] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:19:38,367] - [rest_client:840] ERROR - POST http://127.0.0.1:9001/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:19:38,393] - [task:132] INFO - server: ip:127.0.0.1 port:9002 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_2@cb.local', 'uptime': u'329', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9002', 'storage': [], 'moxi': 11211, 'port': u'9002', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12004, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13433655296, 'memoryTotal': 15466930176, 'memoryQuota': 7906, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:19:38,404] - [rest_client:953] INFO - pools/default params : memoryQuota=7906
[2019-12-10 23:19:38,411] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9002:username=Administrator&password=asdasd&port=9002
[2019-12-10 23:19:38,468] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:19:38,468] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:19:39,362] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9002/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:19:39,508] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:19:39,517] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9002: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:19:39,517] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:19:39,522] - [rest_client:840] ERROR - POST http://127.0.0.1:9002/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:19:39,551] - [task:132] INFO - server: ip:127.0.0.1 port:9003 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_3@cb.local', 'uptime': u'329', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9003', 'storage': [], 'moxi': 11211, 'port': u'9003', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12006, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13435486208, 'memoryTotal': 15466930176, 'memoryQuota': 7906, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:19:39,561] - [rest_client:953] INFO - pools/default params : memoryQuota=7906
[2019-12-10 23:19:39,569] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9003:username=Administrator&password=asdasd&port=9003
[2019-12-10 23:19:39,625] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:19:39,625] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:19:40,529] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9003/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:19:40,672] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:19:40,679] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9003: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:19:40,680] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:19:40,685] - [rest_client:840] ERROR - POST http://127.0.0.1:9003/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:19:40,719] - [basetestcase:2218] INFO - **** add built-in 'cbadminbucket' user to node 127.0.0.1 ****
[2019-12-10 23:19:40,826] - [basetestcase:530] INFO - sleep for 5 secs.  ...
[2019-12-10 23:19:45,831] - [basetestcase:2223] INFO - **** add 'admin' role to 'cbadminbucket' user ****
[2019-12-10 23:19:45,855] - [basetestcase:291] INFO - done initializing cluster
[2019-12-10 23:19:45,859] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:19:45,859] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:19:46,749] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9000/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:19:46,893] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:19:46,894] - [basetestcase:2770] INFO - Enabled diag/eval for non-local hosts from 127.0.0.1
[2019-12-10 23:19:47,747] - [rest_client:2430] INFO - http://127.0.0.1:9000/pools/default/buckets with param: bucketType=membase&evictionPolicy=valueOnly&threadsNumber=3&ramQuotaMB=7650&authType=sasl&name=default&flushEnabled=1&replicaNumber=1&compressionMode=passive&saslPassword=None&replicaIndex=1
[2019-12-10 23:19:47,764] - [rest_client:2455] INFO - 0.02 seconds to create bucket default
[2019-12-10 23:19:47,765] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:19:47,936] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:19:48,021] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:19:48,062] - [task:351] WARNING - vbucket map not ready after try 0
[2019-12-10 23:19:48,062] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:19:48,146] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:19:48,223] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:19:48,265] - [task:351] WARNING - vbucket map not ready after try 1
[2019-12-10 23:19:48,265] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:19:48,351] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:19:48,431] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:19:48,474] - [task:351] WARNING - vbucket map not ready after try 2
[2019-12-10 23:19:48,474] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:19:48,718] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:19:48,795] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:19:48,836] - [task:351] WARNING - vbucket map not ready after try 3
[2019-12-10 23:19:48,837] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:19:48,914] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:19:49,079] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:19:49,122] - [task:351] WARNING - vbucket map not ready after try 4
[2019-12-10 23:19:49,123] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:19:49,200] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:19:49,278] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:19:49,321] - [task:351] WARNING - vbucket map not ready after try 5
[2019-12-10 23:19:49,367] - [basetestcase:383] INFO - ==============  basetestcase setup was finished for test #10 test_multi_create_drop_index ==============
[2019-12-10 23:19:49,392] - [basetestcase:397] INFO - ------- Cluster statistics -------
[2019-12-10 23:19:49,393] - [basetestcase:399] INFO - 127.0.0.1:9000 => {'swap_mem_used': 0, 'cpu_utilization': 12.02046035805627, 'mem_free': 13440069632, 'swap_mem_total': 0, 'mem_total': 15466930176, 'services': [u'index', u'kv', u'n1ql']}
[2019-12-10 23:19:49,393] - [basetestcase:400] INFO - --- End of cluster statistics ---
[2019-12-10 23:19:49,405] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is warmup
[2019-12-10 23:19:49,406] - [newtuq:22] INFO - Waiting for cluster to become healthy
[2019-12-10 23:19:49,406] - [basetestcase:530] INFO - sleep for 5 secs.  ...
[2019-12-10 23:19:54,417] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:19:54,430] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is healthy
[2019-12-10 23:19:54,431] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:19:54,444] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is healthy
[2019-12-10 23:19:54,444] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:19:54,457] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is healthy
[2019-12-10 23:19:54,458] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:19:54,470] - [basetestcase:2436] INFO - list of index nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:19:54,492] - [rest_client:1706] INFO - {'indexer.settings.storage_mode': 'forestdb'} set
[2019-12-10 23:19:54,529] - [rest_client:1706] INFO - {'indexer.api.enableTestServer': True} set
[2019-12-10 23:19:54,533] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:19:54,533] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:19:56,797] - [basetestcase:2545] INFO - create 2016 to default documents...
[2019-12-10 23:19:56,850] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:19:58,633] - [basetestcase:2557] INFO - LOAD IS FINISHED
[2019-12-10 23:19:58,673] - [basetestcase:2436] INFO - list of n1ql nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:19:58,674] - [newtuq:83] INFO - ip:127.0.0.1 port:9000 ssh_username:Administrator
[2019-12-10 23:19:58,674] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2019-12-10 23:19:58,681] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2019-12-10 23:19:58,773] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 88.771898ms
[2019-12-10 23:19:58,773] - [tuq_helper:147] INFO - RUN QUERY CREATE PRIMARY INDEX ON default  USING VIEW 
[2019-12-10 23:19:58,784] - [rest_client:3465] INFO - query params : statement=CREATE+PRIMARY+INDEX+ON+default++USING+VIEW+
[2019-12-10 23:19:59,253] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 466.346929ms
[2019-12-10 23:19:59,253] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2019-12-10 23:19:59,261] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2019-12-10 23:19:59,275] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 12.139145ms
[2019-12-10 23:19:59,290] - [base_2i:153] INFO - []
[2019-12-10 23:19:59,290] - [basetestcase:530] INFO - sleep for 3 secs.  ...
[2019-12-10 23:20:00,291] - [tuq_helper:147] INFO - RUN QUERY CREATE INDEX employee50b2829b107f4ea097c430bd03f95170join_yr ON default(join_yr) USING GSI  WITH {'defer_build': True}
[2019-12-10 23:20:00,298] - [rest_client:3465] INFO - query params : statement=CREATE+INDEX+employee50b2829b107f4ea097c430bd03f95170join_yr+ON+default%28join_yr%29+USING+GSI++WITH+%7B%27defer_build%27%3A+True%7D
[2019-12-10 23:20:00,545] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 244.357826ms
[2019-12-10 23:20:02,292] - [basetestcase:530] INFO - sleep for 3 secs.  ...
[2019-12-10 23:20:02,547] - [tuq_helper:147] INFO - RUN QUERY CREATE INDEX employee50b2829b107f4ea097c430bd03f95170job_title ON default(job_title) USING GSI  WITH {'defer_build': True}
[2019-12-10 23:20:02,555] - [rest_client:3465] INFO - query params : statement=CREATE+INDEX+employee50b2829b107f4ea097c430bd03f95170job_title+ON+default%28job_title%29+USING+GSI++WITH+%7B%27defer_build%27%3A+True%7D
[2019-12-10 23:20:02,848] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 289.558845ms
[2019-12-10 23:20:05,293] - [base_2i:109] INFO - BUILD INDEX on default(employee50b2829b107f4ea097c430bd03f95170join_yr,employee50b2829b107f4ea097c430bd03f95170job_title) USING GSI
[2019-12-10 23:20:05,850] - [tuq_helper:147] INFO - RUN QUERY BUILD INDEX on default(employee50b2829b107f4ea097c430bd03f95170join_yr,employee50b2829b107f4ea097c430bd03f95170job_title) USING GSI
[2019-12-10 23:20:05,858] - [rest_client:3465] INFO - query params : statement=BUILD+INDEX+on+default%28employee50b2829b107f4ea097c430bd03f95170join_yr%2Cemployee50b2829b107f4ea097c430bd03f95170job_title%29+USING+GSI
[2019-12-10 23:20:05,959] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 97.858049ms
[2019-12-10 23:20:06,960] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee50b2829b107f4ea097c430bd03f95170join_yr'
[2019-12-10 23:20:06,967] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee50b2829b107f4ea097c430bd03f95170join_yr%27
[2019-12-10 23:20:06,987] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 17.13104ms
[2019-12-10 23:20:06,987] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee50b2829b107f4ea097c430bd03f95170job_title'
[2019-12-10 23:20:06,994] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee50b2829b107f4ea097c430bd03f95170job_title%27
[2019-12-10 23:20:07,008] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 11.121881ms
[2019-12-10 23:20:07,009] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee50b2829b107f4ea097c430bd03f95170join_yr'
[2019-12-10 23:20:07,016] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee50b2829b107f4ea097c430bd03f95170join_yr%27
[2019-12-10 23:20:07,031] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 12.164692ms
[2019-12-10 23:20:07,031] - [tuq_helper:147] INFO - RUN QUERY DROP INDEX default.employee50b2829b107f4ea097c430bd03f95170join_yr USING GSI
[2019-12-10 23:20:07,039] - [rest_client:3465] INFO - query params : statement=DROP+INDEX+default.employee50b2829b107f4ea097c430bd03f95170join_yr+USING+GSI
[2019-12-10 23:20:07,109] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 67.380834ms
[2019-12-10 23:20:07,110] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee50b2829b107f4ea097c430bd03f95170job_title'
[2019-12-10 23:20:07,119] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee50b2829b107f4ea097c430bd03f95170job_title%27
[2019-12-10 23:20:07,139] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 16.904768ms
[2019-12-10 23:20:07,140] - [tuq_helper:147] INFO - RUN QUERY DROP INDEX default.employee50b2829b107f4ea097c430bd03f95170job_title USING GSI
[2019-12-10 23:20:07,150] - [rest_client:3465] INFO - query params : statement=DROP+INDEX+default.employee50b2829b107f4ea097c430bd03f95170job_title+USING+GSI
[2019-12-10 23:20:07,238] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 85.024455ms
[2019-12-10 23:20:07,239] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee50b2829b107f4ea097c430bd03f95170join_yr'
[2019-12-10 23:20:07,254] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee50b2829b107f4ea097c430bd03f95170join_yr%27
[2019-12-10 23:20:07,276] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 19.178357ms
[2019-12-10 23:20:07,277] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee50b2829b107f4ea097c430bd03f95170job_title'
[2019-12-10 23:20:07,293] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee50b2829b107f4ea097c430bd03f95170job_title%27
[2019-12-10 23:20:07,308] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 12.263161ms
[2019-12-10 23:20:07,334] - [basetestcase:2436] INFO - list of index nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:20:07,337] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:20:07,338] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:20:08,261] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9000: True content: "/opt/build/ns_server/logs/n_0" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))).
[2019-12-10 23:20:08,262] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: zgrep "panic" "/opt/build/ns_server/logs/n_0"/indexer.log* | wc -l
[2019-12-10 23:20:08,399] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:20:08,416] - [basetestcase:2436] INFO - list of kv nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:20:08,420] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:20:08,420] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:20:09,327] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9000: True content: "/opt/build/ns_server/logs/n_0" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))).
[2019-12-10 23:20:09,328] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: zgrep "panic" "/opt/build/ns_server/logs/n_0"/projector.log* | wc -l
[2019-12-10 23:20:09,464] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:20:09,482] - [basetestcase:2436] INFO - list of n1ql nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:20:09,482] - [tuq_helper:516] INFO - CHECK FOR PRIMARY INDEXES
[2019-12-10 23:20:09,482] - [tuq_helper:523] INFO - DROP PRIMARY INDEX ON default USING VIEW 
[2019-12-10 23:20:09,483] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2019-12-10 23:20:09,494] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2019-12-10 23:20:09,511] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 14.147611ms
[2019-12-10 23:20:09,511] - [tuq_helper:147] INFO - RUN QUERY DROP PRIMARY INDEX ON default USING VIEW 
[2019-12-10 23:20:09,518] - [rest_client:3465] INFO - query params : statement=DROP+PRIMARY+INDEX+ON+default+USING+VIEW+
[2019-12-10 23:20:09,534] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 12.778829ms
[2019-12-10 23:20:09,535] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: rm -f /opt/couchbase/var/lib/couchbase/meta;rm -f /tmp/log_upr_client.sock
[2019-12-10 23:20:09,662] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:20:09,663] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: killall cbq-engine
[2019-12-10 23:20:09,811] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:20:09,812] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: killall tuqtng
[2019-12-10 23:20:09,984] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:20:09,985] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: killall indexer
[2019-12-10 23:20:10,154] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:20:10,189] - [basetestcase:397] INFO - ------- Cluster statistics -------
[2019-12-10 23:20:10,189] - [basetestcase:399] INFO - 127.0.0.1:9000 => {'swap_mem_used': 0, 'cpu_utilization': 6.741573033707865, 'mem_free': 13375242240, 'swap_mem_total': 0, 'mem_total': 15466930176, 'services': [u'index', u'kv', u'n1ql']}
[2019-12-10 23:20:10,190] - [basetestcase:400] INFO - --- End of cluster statistics ---
[2019-12-10 23:20:10,190] - [basetestcase:462] INFO - ==============  basetestcase cleanup was started for test #10 test_multi_create_drop_index ==============
[2019-12-10 23:20:10,241] - [bucket_helper:143] INFO - deleting existing buckets [u'default'] on 127.0.0.1
[2019-12-10 23:20:10,241] - [bucket_helper:145] INFO - remove bucket default ...
[2019-12-10 23:20:11,088] - [bucket_helper:159] INFO - deleted bucket : default from 127.0.0.1
[2019-12-10 23:20:11,088] - [bucket_helper:237] INFO - waiting for bucket deletion to complete....
[2019-12-10 23:20:11,096] - [rest_client:133] INFO - node 127.0.0.1 existing buckets : []
[2019-12-10 23:20:11,097] - [bucket_helper:169] INFO - sleep 2 seconds to make sure all buckets ([u'default']) were deleted completely.
[2019-12-10 23:20:13,168] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9000
[2019-12-10 23:20:13,174] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9000 is running
[2019-12-10 23:20:13,181] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9001
[2019-12-10 23:20:13,187] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9001 is running
[2019-12-10 23:20:13,194] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9002
[2019-12-10 23:20:13,200] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9002 is running
[2019-12-10 23:20:13,206] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9003
[2019-12-10 23:20:13,212] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9003 is running
[2019-12-10 23:20:13,212] - [basetestcase:480] INFO - ==============  basetestcase cleanup was finished for test #10 test_multi_create_drop_index ==============
ok

----------------------------------------------------------------------
Ran 1 test in 39.200s

OK
test_multi_create_drop_index (2i.indexcreatedrop_2i.SecondaryIndexingCreateDropTests) ... Cluster instance shutdown with force
summary so far suite 2i.indexscans_2i.SecondaryIndexingScanTests , pass 9 , fail 0
summary so far suite 2i.indexcreatedrop_2i.SecondaryIndexingCreateDropTests , pass 1 , fail 0
testrunner logs, diags and results are available under /opt/build/testrunner/logs/testrunner-19-Dec-10_23-14-02/test_10
Logs will be stored at /opt/build/testrunner/logs/testrunner-19-Dec-10_23-14-02/test_11

./testrunner -i b/resources/dev-4-nodes-xdcr_n1ql_2i.ini -p makefile=True -t 2i.indexcreatedrop_2i.SecondaryIndexingCreateDropTests.test_multi_create_drop_index,groups=composite,dataset=default,doc-per-day=1,cbq_version=sherlock,skip_build_tuq=True,GROUP=views

Test Input params:
{'doc-per-day': '1', 'logs_folder': '/opt/build/testrunner/logs/testrunner-19-Dec-10_23-14-02/test_11', 'GROUP': 'views', 'gsi_type': 'forestdb', 'makefile': 'True', 'skip_build_tuq': 'True', 'conf_file': 'conf/simple_2i_n1ql.conf', 'dataset': 'default', 'cluster_name': 'dev-4-nodes-xdcr_n1ql_2i', 'ini': 'b/resources/dev-4-nodes-xdcr_n1ql_2i.ini', 'groups': 'composite', 'case_number': 11, 'num_nodes': 4, 'spec': 'simple_2i_n1ql', 'cbq_version': 'sherlock'}
[2019-12-10 23:20:13,271] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:20:13,271] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:20:14,217] - [rest_client:2127] INFO - Node version in cluster 6.5.0-4928-rel-enterprise
[2019-12-10 23:20:14,231] - [rest_client:2137] INFO - Node versions in cluster [u'6.5.0-4928-rel-enterprise']
[2019-12-10 23:20:14,231] - [basetestcase:218] INFO - ==============  basetestcase setup was started for test #11 test_multi_create_drop_index==============
[2019-12-10 23:20:14,325] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9000
[2019-12-10 23:20:14,331] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9000 is running
[2019-12-10 23:20:14,338] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9001
[2019-12-10 23:20:14,344] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9001 is running
[2019-12-10 23:20:14,350] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9002
[2019-12-10 23:20:14,357] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9002 is running
[2019-12-10 23:20:14,363] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9003
[2019-12-10 23:20:14,369] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9003 is running
[2019-12-10 23:20:14,370] - [basetestcase:265] INFO - initializing cluster
[2019-12-10 23:20:15,286] - [task:132] INFO - server: ip:127.0.0.1 port:9000 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_0@cb.local', 'uptime': u'369', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9000', 'storage': [], 'moxi': 11211, 'port': u'9000', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12000, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'index', u'kv', u'n1ql'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13429932032, 'memoryTotal': 15466930176, 'memoryQuota': 7650, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:20:15,297] - [task:171] INFO - quota for index service will be 256 MB
[2019-12-10 23:20:15,297] - [task:173] INFO - set index quota to node 127.0.0.1 
[2019-12-10 23:20:15,297] - [rest_client:965] INFO - pools/default params : indexMemoryQuota=256
[2019-12-10 23:20:15,313] - [rest_client:953] INFO - pools/default params : memoryQuota=7650
[2019-12-10 23:20:15,329] - [rest_client:931] INFO - /node/controller/setupServices params on 127.0.0.1:9000:services=kv%2Cindex%2Cn1ql&password=asdasd&user=Administrator&hostname=127.0.0.1%3A9000
[2019-12-10 23:20:15,340] - [rest_client:840] ERROR - POST http://127.0.0.1:9000//node/controller/setupServices body: services=kv%2Cindex%2Cn1ql&password=asdasd&user=Administrator&hostname=127.0.0.1%3A9000 headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown ["cannot change node services after cluster is provisioned"] auth: Administrator:asdasd
[2019-12-10 23:20:15,341] - [rest_client:936] INFO - This node is already provisioned with services, we do not consider this as failure for test case
[2019-12-10 23:20:15,341] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9000:username=Administrator&password=asdasd&port=9000
[2019-12-10 23:20:15,401] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:20:15,401] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:20:16,325] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9000/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:20:16,467] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:20:16,474] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9000: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:20:16,475] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:20:16,481] - [rest_client:840] ERROR - POST http://127.0.0.1:9000/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:20:16,512] - [task:132] INFO - server: ip:127.0.0.1 port:9001 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_1@cb.local', 'uptime': u'369', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9001', 'storage': [], 'moxi': 11211, 'port': u'9001', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12002, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13429932032, 'memoryTotal': 15466930176, 'memoryQuota': 7906, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:20:16,522] - [rest_client:953] INFO - pools/default params : memoryQuota=7906
[2019-12-10 23:20:16,531] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9001:username=Administrator&password=asdasd&port=9001
[2019-12-10 23:20:16,586] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:20:16,586] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:20:17,491] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9001/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:20:17,635] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:20:17,643] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9001: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:20:17,643] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:20:17,650] - [rest_client:840] ERROR - POST http://127.0.0.1:9001/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:20:17,677] - [task:132] INFO - server: ip:127.0.0.1 port:9002 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_2@cb.local', 'uptime': u'369', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9002', 'storage': [], 'moxi': 11211, 'port': u'9002', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12004, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13430325248, 'memoryTotal': 15466930176, 'memoryQuota': 7906, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:20:17,687] - [rest_client:953] INFO - pools/default params : memoryQuota=7906
[2019-12-10 23:20:17,696] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9002:username=Administrator&password=asdasd&port=9002
[2019-12-10 23:20:17,757] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:20:17,757] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:20:18,652] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9002/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:20:18,798] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:20:18,806] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9002: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:20:18,807] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:20:18,813] - [rest_client:840] ERROR - POST http://127.0.0.1:9002/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:20:18,839] - [task:132] INFO - server: ip:127.0.0.1 port:9003 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_3@cb.local', 'uptime': u'369', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9003', 'storage': [], 'moxi': 11211, 'port': u'9003', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12006, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13430325248, 'memoryTotal': 15466930176, 'memoryQuota': 7906, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:20:18,848] - [rest_client:953] INFO - pools/default params : memoryQuota=7906
[2019-12-10 23:20:18,855] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9003:username=Administrator&password=asdasd&port=9003
[2019-12-10 23:20:18,916] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:20:18,916] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:20:19,811] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9003/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:20:19,955] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:20:19,965] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9003: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:20:19,965] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:20:19,970] - [rest_client:840] ERROR - POST http://127.0.0.1:9003/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:20:20,001] - [basetestcase:2218] INFO - **** add built-in 'cbadminbucket' user to node 127.0.0.1 ****
[2019-12-10 23:20:20,113] - [basetestcase:530] INFO - sleep for 5 secs.  ...
[2019-12-10 23:20:25,116] - [basetestcase:2223] INFO - **** add 'admin' role to 'cbadminbucket' user ****
[2019-12-10 23:20:25,139] - [basetestcase:291] INFO - done initializing cluster
[2019-12-10 23:20:25,144] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:20:25,144] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:20:26,062] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9000/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:20:26,206] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:20:26,207] - [basetestcase:2770] INFO - Enabled diag/eval for non-local hosts from 127.0.0.1
[2019-12-10 23:20:27,048] - [rest_client:2430] INFO - http://127.0.0.1:9000/pools/default/buckets with param: bucketType=membase&evictionPolicy=valueOnly&threadsNumber=3&ramQuotaMB=7650&authType=sasl&name=default&flushEnabled=1&replicaNumber=1&compressionMode=passive&saslPassword=None&replicaIndex=1
[2019-12-10 23:20:27,060] - [rest_client:2455] INFO - 0.01 seconds to create bucket default
[2019-12-10 23:20:27,061] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:20:27,218] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:20:27,300] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:20:27,344] - [task:351] WARNING - vbucket map not ready after try 0
[2019-12-10 23:20:27,344] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:20:27,614] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:20:27,692] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:20:27,739] - [task:351] WARNING - vbucket map not ready after try 1
[2019-12-10 23:20:27,739] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:20:27,817] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:20:27,897] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:20:28,031] - [task:351] WARNING - vbucket map not ready after try 2
[2019-12-10 23:20:28,031] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:20:28,115] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:20:28,220] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:20:28,264] - [task:351] WARNING - vbucket map not ready after try 3
[2019-12-10 23:20:28,264] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:20:28,339] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:20:28,414] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:20:28,551] - [task:351] WARNING - vbucket map not ready after try 4
[2019-12-10 23:20:28,551] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:20:28,626] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:20:28,704] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:20:28,746] - [task:351] WARNING - vbucket map not ready after try 5
[2019-12-10 23:20:28,790] - [basetestcase:383] INFO - ==============  basetestcase setup was finished for test #11 test_multi_create_drop_index ==============
[2019-12-10 23:20:28,816] - [basetestcase:397] INFO - ------- Cluster statistics -------
[2019-12-10 23:20:28,816] - [basetestcase:399] INFO - 127.0.0.1:9000 => {'swap_mem_used': 0, 'cpu_utilization': 22.16494845360825, 'mem_free': 13433524224, 'swap_mem_total': 0, 'mem_total': 15466930176, 'services': [u'index', u'kv', u'n1ql']}
[2019-12-10 23:20:28,816] - [basetestcase:400] INFO - --- End of cluster statistics ---
[2019-12-10 23:20:28,828] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is warmup
[2019-12-10 23:20:28,829] - [newtuq:22] INFO - Waiting for cluster to become healthy
[2019-12-10 23:20:28,829] - [basetestcase:530] INFO - sleep for 5 secs.  ...
[2019-12-10 23:20:33,841] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:20:33,854] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is healthy
[2019-12-10 23:20:33,855] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:20:33,868] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is healthy
[2019-12-10 23:20:33,868] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:20:33,882] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is healthy
[2019-12-10 23:20:33,882] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:20:33,895] - [basetestcase:2436] INFO - list of index nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:20:33,915] - [rest_client:1706] INFO - {'indexer.settings.storage_mode': 'forestdb'} set
[2019-12-10 23:20:33,951] - [rest_client:1706] INFO - {'indexer.api.enableTestServer': True} set
[2019-12-10 23:20:33,955] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:20:33,955] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:20:36,268] - [basetestcase:2545] INFO - create 2016 to default documents...
[2019-12-10 23:20:36,319] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:20:38,045] - [basetestcase:2557] INFO - LOAD IS FINISHED
[2019-12-10 23:20:38,085] - [basetestcase:2436] INFO - list of n1ql nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:20:38,086] - [newtuq:83] INFO - ip:127.0.0.1 port:9000 ssh_username:Administrator
[2019-12-10 23:20:38,086] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2019-12-10 23:20:38,094] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2019-12-10 23:20:38,184] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 86.934847ms
[2019-12-10 23:20:38,184] - [tuq_helper:147] INFO - RUN QUERY CREATE PRIMARY INDEX ON default  USING VIEW 
[2019-12-10 23:20:38,193] - [rest_client:3465] INFO - query params : statement=CREATE+PRIMARY+INDEX+ON+default++USING+VIEW+
[2019-12-10 23:20:38,681] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 484.938607ms
[2019-12-10 23:20:38,681] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2019-12-10 23:20:38,688] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2019-12-10 23:20:38,703] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 12.65632ms
[2019-12-10 23:20:38,718] - [base_2i:153] INFO - []
[2019-12-10 23:20:38,719] - [basetestcase:530] INFO - sleep for 3 secs.  ...
[2019-12-10 23:20:39,719] - [tuq_helper:147] INFO - RUN QUERY CREATE INDEX employeeb66ae923b28e4749a660ca71c2e380b0job_title_join_yr ON default(join_yr,job_title) USING GSI  WITH {'defer_build': True}
[2019-12-10 23:20:39,726] - [rest_client:3465] INFO - query params : statement=CREATE+INDEX+employeeb66ae923b28e4749a660ca71c2e380b0job_title_join_yr+ON+default%28join_yr%2Cjob_title%29+USING+GSI++WITH+%7B%27defer_build%27%3A+True%7D
[2019-12-10 23:20:39,982] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 252.765608ms
[2019-12-10 23:20:41,722] - [base_2i:109] INFO - BUILD INDEX on default(employeeb66ae923b28e4749a660ca71c2e380b0job_title_join_yr) USING GSI
[2019-12-10 23:20:41,983] - [tuq_helper:147] INFO - RUN QUERY BUILD INDEX on default(employeeb66ae923b28e4749a660ca71c2e380b0job_title_join_yr) USING GSI
[2019-12-10 23:20:41,991] - [rest_client:3465] INFO - query params : statement=BUILD+INDEX+on+default%28employeeb66ae923b28e4749a660ca71c2e380b0job_title_join_yr%29+USING+GSI
[2019-12-10 23:20:42,078] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 83.744578ms
[2019-12-10 23:20:43,079] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employeeb66ae923b28e4749a660ca71c2e380b0job_title_join_yr'
[2019-12-10 23:20:43,087] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employeeb66ae923b28e4749a660ca71c2e380b0job_title_join_yr%27
[2019-12-10 23:20:43,105] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 15.573813ms
[2019-12-10 23:20:44,106] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employeeb66ae923b28e4749a660ca71c2e380b0job_title_join_yr'
[2019-12-10 23:20:44,114] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employeeb66ae923b28e4749a660ca71c2e380b0job_title_join_yr%27
[2019-12-10 23:20:44,132] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 16.141872ms
[2019-12-10 23:20:44,133] - [tuq_helper:147] INFO - RUN QUERY DROP INDEX default.employeeb66ae923b28e4749a660ca71c2e380b0job_title_join_yr USING GSI
[2019-12-10 23:20:44,139] - [rest_client:3465] INFO - query params : statement=DROP+INDEX+default.employeeb66ae923b28e4749a660ca71c2e380b0job_title_join_yr+USING+GSI
[2019-12-10 23:20:44,219] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 75.996827ms
[2019-12-10 23:20:44,219] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employeeb66ae923b28e4749a660ca71c2e380b0job_title_join_yr'
[2019-12-10 23:20:44,233] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employeeb66ae923b28e4749a660ca71c2e380b0job_title_join_yr%27
[2019-12-10 23:20:44,256] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 20.169778ms
[2019-12-10 23:20:44,284] - [basetestcase:2436] INFO - list of index nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:20:44,288] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:20:44,288] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:20:45,256] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9000: True content: "/opt/build/ns_server/logs/n_0" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))).
[2019-12-10 23:20:45,257] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: zgrep "panic" "/opt/build/ns_server/logs/n_0"/indexer.log* | wc -l
[2019-12-10 23:20:45,399] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:20:45,417] - [basetestcase:2436] INFO - list of kv nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:20:45,421] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:20:45,421] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:20:46,390] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9000: True content: "/opt/build/ns_server/logs/n_0" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))).
[2019-12-10 23:20:46,390] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: zgrep "panic" "/opt/build/ns_server/logs/n_0"/projector.log* | wc -l
[2019-12-10 23:20:46,538] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:20:46,556] - [basetestcase:2436] INFO - list of n1ql nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:20:46,556] - [tuq_helper:516] INFO - CHECK FOR PRIMARY INDEXES
[2019-12-10 23:20:46,557] - [tuq_helper:523] INFO - DROP PRIMARY INDEX ON default USING VIEW 
[2019-12-10 23:20:46,557] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2019-12-10 23:20:46,564] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2019-12-10 23:20:46,578] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 11.413186ms
[2019-12-10 23:20:46,578] - [tuq_helper:147] INFO - RUN QUERY DROP PRIMARY INDEX ON default USING VIEW 
[2019-12-10 23:20:46,585] - [rest_client:3465] INFO - query params : statement=DROP+PRIMARY+INDEX+ON+default+USING+VIEW+
[2019-12-10 23:20:46,597] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 8.372515ms
[2019-12-10 23:20:46,598] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: rm -f /opt/couchbase/var/lib/couchbase/meta;rm -f /tmp/log_upr_client.sock
[2019-12-10 23:20:46,736] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:20:46,737] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: killall cbq-engine
[2019-12-10 23:20:46,873] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:20:46,875] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: killall tuqtng
[2019-12-10 23:20:47,051] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:20:47,052] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: killall indexer
[2019-12-10 23:20:47,225] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:20:47,247] - [basetestcase:397] INFO - ------- Cluster statistics -------
[2019-12-10 23:20:47,247] - [basetestcase:399] INFO - 127.0.0.1:9000 => {'swap_mem_used': 0, 'cpu_utilization': 18.76750700280112, 'mem_free': 13344210944, 'swap_mem_total': 0, 'mem_total': 15466930176, 'services': [u'index', u'kv', u'n1ql']}
[2019-12-10 23:20:47,247] - [basetestcase:400] INFO - --- End of cluster statistics ---
[2019-12-10 23:20:47,248] - [basetestcase:462] INFO - ==============  basetestcase cleanup was started for test #11 test_multi_create_drop_index ==============
[2019-12-10 23:20:47,300] - [bucket_helper:143] INFO - deleting existing buckets [u'default'] on 127.0.0.1
[2019-12-10 23:20:47,301] - [bucket_helper:145] INFO - remove bucket default ...
[2019-12-10 23:20:48,368] - [bucket_helper:159] INFO - deleted bucket : default from 127.0.0.1
[2019-12-10 23:20:48,368] - [bucket_helper:237] INFO - waiting for bucket deletion to complete....
[2019-12-10 23:20:48,374] - [rest_client:133] INFO - node 127.0.0.1 existing buckets : []
[2019-12-10 23:20:48,374] - [bucket_helper:169] INFO - sleep 2 seconds to make sure all buckets ([u'default']) were deleted completely.
[2019-12-10 23:20:50,442] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9000
[2019-12-10 23:20:50,449] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9000 is running
[2019-12-10 23:20:50,456] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9001
[2019-12-10 23:20:50,462] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9001 is running
[2019-12-10 23:20:50,468] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9002
[2019-12-10 23:20:50,475] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9002 is running
[2019-12-10 23:20:50,481] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9003
[2019-12-10 23:20:50,488] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9003 is running
[2019-12-10 23:20:50,489] - [basetestcase:480] INFO - ==============  basetestcase cleanup was finished for test #11 test_multi_create_drop_index ==============
ok

----------------------------------------------------------------------
Ran 1 test in 37.236s

OK
test_remove_bucket_and_query (2i.cluster_ops_2i.SecondaryIndexingClusterOpsTests) ... Cluster instance shutdown with force
summary so far suite 2i.indexscans_2i.SecondaryIndexingScanTests , pass 9 , fail 0
summary so far suite 2i.indexcreatedrop_2i.SecondaryIndexingCreateDropTests , pass 2 , fail 0
testrunner logs, diags and results are available under /opt/build/testrunner/logs/testrunner-19-Dec-10_23-14-02/test_11
Logs will be stored at /opt/build/testrunner/logs/testrunner-19-Dec-10_23-14-02/test_12

./testrunner -i b/resources/dev-4-nodes-xdcr_n1ql_2i.ini -p makefile=True -t 2i.cluster_ops_2i.SecondaryIndexingClusterOpsTests.test_remove_bucket_and_query,groups=simple:and:no_orderby_groupby:range,dataset=default,doc-per-day=1,use_gsi_for_primary=True,use_gsi_for_secondary=True,GROUP=gsi

Test Input params:
{'doc-per-day': '1', 'use_gsi_for_primary': 'True', 'GROUP': 'gsi', 'use_gsi_for_secondary': 'True', 'gsi_type': 'forestdb', 'makefile': 'True', 'conf_file': 'conf/simple_2i_n1ql.conf', 'dataset': 'default', 'cluster_name': 'dev-4-nodes-xdcr_n1ql_2i', 'ini': 'b/resources/dev-4-nodes-xdcr_n1ql_2i.ini', 'groups': 'simple:and:no_orderby_groupby:range', 'case_number': 12, 'num_nodes': 4, 'logs_folder': '/opt/build/testrunner/logs/testrunner-19-Dec-10_23-14-02/test_12', 'spec': 'simple_2i_n1ql'}
[2019-12-10 23:20:50,552] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:20:50,553] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:20:51,525] - [rest_client:2127] INFO - Node version in cluster 6.5.0-4928-rel-enterprise
[2019-12-10 23:20:51,539] - [rest_client:2137] INFO - Node versions in cluster [u'6.5.0-4928-rel-enterprise']
[2019-12-10 23:20:51,539] - [basetestcase:218] INFO - ==============  basetestcase setup was started for test #12 test_remove_bucket_and_query==============
[2019-12-10 23:20:51,632] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9000
[2019-12-10 23:20:51,639] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9000 is running
[2019-12-10 23:20:51,645] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9001
[2019-12-10 23:20:51,651] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9001 is running
[2019-12-10 23:20:51,658] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9002
[2019-12-10 23:20:51,664] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9002 is running
[2019-12-10 23:20:51,670] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9003
[2019-12-10 23:20:51,677] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9003 is running
[2019-12-10 23:20:51,678] - [basetestcase:265] INFO - initializing cluster
[2019-12-10 23:20:52,565] - [task:132] INFO - server: ip:127.0.0.1 port:9000 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_0@cb.local', 'uptime': u'404', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9000', 'storage': [], 'moxi': 11211, 'port': u'9000', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12000, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'index', u'kv', u'n1ql'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13371281408, 'memoryTotal': 15466930176, 'memoryQuota': 7650, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:20:52,576] - [task:171] INFO - quota for index service will be 256 MB
[2019-12-10 23:20:52,576] - [task:173] INFO - set index quota to node 127.0.0.1 
[2019-12-10 23:20:52,576] - [rest_client:965] INFO - pools/default params : indexMemoryQuota=256
[2019-12-10 23:20:52,585] - [rest_client:953] INFO - pools/default params : memoryQuota=7650
[2019-12-10 23:20:52,608] - [rest_client:931] INFO - /node/controller/setupServices params on 127.0.0.1:9000:services=kv%2Cindex%2Cn1ql&password=asdasd&user=Administrator&hostname=127.0.0.1%3A9000
[2019-12-10 23:20:52,617] - [rest_client:840] ERROR - POST http://127.0.0.1:9000//node/controller/setupServices body: services=kv%2Cindex%2Cn1ql&password=asdasd&user=Administrator&hostname=127.0.0.1%3A9000 headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown ["cannot change node services after cluster is provisioned"] auth: Administrator:asdasd
[2019-12-10 23:20:52,618] - [rest_client:936] INFO - This node is already provisioned with services, we do not consider this as failure for test case
[2019-12-10 23:20:52,618] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9000:username=Administrator&password=asdasd&port=9000
[2019-12-10 23:20:52,675] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:20:52,676] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:20:53,627] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9000/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:20:53,778] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:20:53,786] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9000: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:20:53,787] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:20:53,792] - [rest_client:840] ERROR - POST http://127.0.0.1:9000/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:20:53,820] - [task:132] INFO - server: ip:127.0.0.1 port:9001 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_1@cb.local', 'uptime': u'404', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9001', 'storage': [], 'moxi': 11211, 'port': u'9001', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12002, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13371154432, 'memoryTotal': 15466930176, 'memoryQuota': 7906, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:20:53,831] - [rest_client:953] INFO - pools/default params : memoryQuota=7906
[2019-12-10 23:20:53,839] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9001:username=Administrator&password=asdasd&port=9001
[2019-12-10 23:20:53,898] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:20:53,898] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:20:54,856] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9001/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:20:55,008] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:20:55,015] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9001: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:20:55,016] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:20:55,022] - [rest_client:840] ERROR - POST http://127.0.0.1:9001/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:20:55,048] - [task:132] INFO - server: ip:127.0.0.1 port:9002 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_2@cb.local', 'uptime': u'404', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9002', 'storage': [], 'moxi': 11211, 'port': u'9002', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12004, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13371154432, 'memoryTotal': 15466930176, 'memoryQuota': 7906, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:20:55,062] - [rest_client:953] INFO - pools/default params : memoryQuota=7906
[2019-12-10 23:20:55,070] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9002:username=Administrator&password=asdasd&port=9002
[2019-12-10 23:20:55,132] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:20:55,132] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:20:56,105] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9002/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:20:56,257] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:20:56,265] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9002: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:20:56,266] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:20:56,271] - [rest_client:840] ERROR - POST http://127.0.0.1:9002/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:20:56,299] - [task:132] INFO - server: ip:127.0.0.1 port:9003 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_3@cb.local', 'uptime': u'409', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9003', 'storage': [], 'moxi': 11211, 'port': u'9003', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12006, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13410910208, 'memoryTotal': 15466930176, 'memoryQuota': 7906, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:20:56,309] - [rest_client:953] INFO - pools/default params : memoryQuota=7906
[2019-12-10 23:20:56,317] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9003:username=Administrator&password=asdasd&port=9003
[2019-12-10 23:20:56,376] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:20:56,376] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:20:57,335] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9003/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:20:57,486] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:20:57,501] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9003: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:20:57,501] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:20:57,508] - [rest_client:840] ERROR - POST http://127.0.0.1:9003/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:20:57,544] - [basetestcase:2218] INFO - **** add built-in 'cbadminbucket' user to node 127.0.0.1 ****
[2019-12-10 23:20:57,652] - [basetestcase:530] INFO - sleep for 5 secs.  ...
[2019-12-10 23:21:02,656] - [basetestcase:2223] INFO - **** add 'admin' role to 'cbadminbucket' user ****
[2019-12-10 23:21:02,679] - [basetestcase:291] INFO - done initializing cluster
[2019-12-10 23:21:02,683] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:21:02,683] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:21:03,665] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9000/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:21:03,819] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:21:03,821] - [basetestcase:2770] INFO - Enabled diag/eval for non-local hosts from 127.0.0.1
[2019-12-10 23:21:04,649] - [rest_client:2430] INFO - http://127.0.0.1:9000/pools/default/buckets with param: bucketType=membase&evictionPolicy=valueOnly&threadsNumber=3&ramQuotaMB=7650&authType=sasl&name=default&flushEnabled=1&replicaNumber=1&compressionMode=passive&saslPassword=None&replicaIndex=1
[2019-12-10 23:21:04,666] - [rest_client:2455] INFO - 0.02 seconds to create bucket default
[2019-12-10 23:21:04,666] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:21:04,827] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:21:04,922] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:21:04,966] - [task:351] WARNING - vbucket map not ready after try 0
[2019-12-10 23:21:04,967] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:21:05,044] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:21:05,322] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:21:05,364] - [task:351] WARNING - vbucket map not ready after try 1
[2019-12-10 23:21:05,364] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:21:05,442] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:21:05,527] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:21:05,574] - [task:351] WARNING - vbucket map not ready after try 2
[2019-12-10 23:21:05,575] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:21:05,764] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:21:05,872] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:21:05,917] - [task:351] WARNING - vbucket map not ready after try 3
[2019-12-10 23:21:05,918] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:21:06,003] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:21:06,081] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:21:06,123] - [task:351] WARNING - vbucket map not ready after try 4
[2019-12-10 23:21:06,124] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:21:06,304] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:21:06,382] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:21:06,423] - [task:351] WARNING - vbucket map not ready after try 5
[2019-12-10 23:21:06,436] - [basetestcase:383] INFO - ==============  basetestcase setup was finished for test #12 test_remove_bucket_and_query ==============
[2019-12-10 23:21:06,459] - [basetestcase:397] INFO - ------- Cluster statistics -------
[2019-12-10 23:21:06,459] - [basetestcase:399] INFO - 127.0.0.1:9000 => {'swap_mem_used': 0, 'cpu_utilization': 11.05398457583548, 'mem_free': 13422436352, 'swap_mem_total': 0, 'mem_total': 15466930176, 'services': [u'index', u'kv', u'n1ql']}
[2019-12-10 23:21:06,459] - [basetestcase:400] INFO - --- End of cluster statistics ---
[2019-12-10 23:21:06,472] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is warmup
[2019-12-10 23:21:06,472] - [newtuq:22] INFO - Waiting for cluster to become healthy
[2019-12-10 23:21:06,472] - [basetestcase:530] INFO - sleep for 5 secs.  ...
[2019-12-10 23:21:11,482] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:21:11,498] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is healthy
[2019-12-10 23:21:11,498] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:21:11,511] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is healthy
[2019-12-10 23:21:11,512] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:21:11,525] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is healthy
[2019-12-10 23:21:11,526] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:21:11,538] - [basetestcase:2436] INFO - list of index nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:21:11,556] - [rest_client:1706] INFO - {'indexer.settings.storage_mode': 'forestdb'} set
[2019-12-10 23:21:11,588] - [rest_client:1706] INFO - {'indexer.api.enableTestServer': True} set
[2019-12-10 23:21:11,592] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:21:11,593] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:21:13,937] - [basetestcase:2545] INFO - create 2016 to default documents...
[2019-12-10 23:21:13,991] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:21:15,788] - [basetestcase:2557] INFO - LOAD IS FINISHED
[2019-12-10 23:21:15,829] - [basetestcase:2436] INFO - list of n1ql nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:21:15,830] - [newtuq:83] INFO - ip:127.0.0.1 port:9000 ssh_username:Administrator
[2019-12-10 23:21:15,830] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2019-12-10 23:21:15,838] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2019-12-10 23:21:15,932] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 90.410696ms
[2019-12-10 23:21:15,932] - [tuq_helper:147] INFO - RUN QUERY CREATE PRIMARY INDEX ON default  USING GSI
[2019-12-10 23:21:15,940] - [rest_client:3465] INFO - query params : statement=CREATE+PRIMARY+INDEX+ON+default++USING+GSI
[2019-12-10 23:21:16,733] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 789.503867ms
[2019-12-10 23:21:16,733] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2019-12-10 23:21:16,743] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2019-12-10 23:21:16,758] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 12.588581ms
[2019-12-10 23:21:16,792] - [basetestcase:2436] INFO - list of n1ql nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:21:17,780] - [tuq_helper:147] INFO - RUN QUERY CREATE INDEX employee33634bff2658421884081898830f463cjoin_yr ON default(join_yr) USING GSI  WITH {'defer_build': True}
[2019-12-10 23:21:17,787] - [rest_client:3465] INFO - query params : statement=CREATE+INDEX+employee33634bff2658421884081898830f463cjoin_yr+ON+default%28join_yr%29+USING+GSI++WITH+%7B%27defer_build%27%3A+True%7D
[2019-12-10 23:21:18,080] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 289.455258ms
[2019-12-10 23:21:18,080] - [base_2i:109] INFO - BUILD INDEX on default(employee33634bff2658421884081898830f463cjoin_yr) USING GSI
[2019-12-10 23:21:19,081] - [tuq_helper:147] INFO - RUN QUERY BUILD INDEX on default(employee33634bff2658421884081898830f463cjoin_yr) USING GSI
[2019-12-10 23:21:19,089] - [rest_client:3465] INFO - query params : statement=BUILD+INDEX+on+default%28employee33634bff2658421884081898830f463cjoin_yr%29+USING+GSI
[2019-12-10 23:21:19,161] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 68.068902ms
[2019-12-10 23:21:19,161] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee33634bff2658421884081898830f463cjoin_yr'
[2019-12-10 23:21:19,171] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee33634bff2658421884081898830f463cjoin_yr%27
[2019-12-10 23:21:19,187] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 13.477666ms
[2019-12-10 23:21:20,189] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee33634bff2658421884081898830f463cjoin_yr'
[2019-12-10 23:21:20,197] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee33634bff2658421884081898830f463cjoin_yr%27
[2019-12-10 23:21:20,213] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 14.101294ms
[2019-12-10 23:21:21,214] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee33634bff2658421884081898830f463cjoin_yr'
[2019-12-10 23:21:21,222] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee33634bff2658421884081898830f463cjoin_yr%27
[2019-12-10 23:21:21,235] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 10.741509ms
[2019-12-10 23:21:22,236] - [tuq_helper:147] INFO - RUN QUERY EXPLAIN SELECT * FROM default WHERE  join_yr > 2010 and join_yr < 2014 ORDER BY _id 
[2019-12-10 23:21:22,245] - [rest_client:3465] INFO - query params : statement=EXPLAIN+SELECT+%2A+FROM+default+WHERE++join_yr+%3E+2010+and+join_yr+%3C+2014+ORDER+BY+_id+
[2019-12-10 23:21:22,252] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 5.059321ms
[2019-12-10 23:21:22,252] - [base_2i:261] INFO - {u'status': u'success', u'metrics': {u'elapsedTime': u'5.059321ms', u'executionTime': u'5.009788ms', u'resultSize': 895, u'resultCount': 1}, u'results': [{u'text': u'SELECT * FROM default WHERE  join_yr > 2010 and join_yr < 2014 ORDER BY _id', u'plan': {u'#operator': u'Sequence', u'~children': [{u'#operator': u'Sequence', u'~children': [{u'index': u'employee33634bff2658421884081898830f463cjoin_yr', u'namespace': u'default', u'keyspace': u'default', u'#operator': u'IndexScan3', u'index_projection': {u'primary_key': True}, u'spans': [{u'range': [{u'high': u'2014', u'low': u'2010', u'inclusion': 0}], u'exact': True}], u'using': u'gsi', u'index_id': u'8a6be53100ee5cbd'}, {u'keyspace': u'default', u'#operator': u'Fetch', u'namespace': u'default'}, {u'#operator': u'Parallel', u'~child': {u'#operator': u'Sequence', u'~children': [{u'#operator': u'Filter', u'condition': u'((2010 < (`default`.`join_yr`)) and ((`default`.`join_yr`) < 2014))'}, {u'#operator': u'InitialProject', u'result_terms': [{u'expr': u'self', u'star': True}]}]}}]}, {u'#operator': u'Order', u'sort_terms': [{u'expr': u'(`default`.`_id`)'}]}, {u'#operator': u'FinalProject'}]}}], u'requestID': u'34e78c2d-8a00-45af-9f00-6fc46a75a334', u'signature': u'json'}
[2019-12-10 23:21:22,253] - [tuq_generators:70] INFO - FROM clause ===== is default
[2019-12-10 23:21:22,254] - [tuq_generators:72] INFO - WHERE clause ===== is   doc["join_yr"] > 2010 and doc["join_yr"] < 2014 
[2019-12-10 23:21:22,254] - [tuq_generators:74] INFO - UNNEST clause ===== is None
[2019-12-10 23:21:22,254] - [tuq_generators:76] INFO - SELECT clause ===== is {"*" : doc,}
[2019-12-10 23:21:22,255] - [tuq_generators:417] INFO - ORDER clause ========= is doc["_id"],
[2019-12-10 23:21:22,256] - [tuq_generators:417] INFO - ORDER clause ========= is doc["_id"],
[2019-12-10 23:21:22,347] - [tuq_generators:417] INFO - ORDER clause ========= is doc["_id"],
[2019-12-10 23:21:22,361] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM default WHERE  join_yr > 2010 and join_yr < 2014 ORDER BY _id 
[2019-12-10 23:21:22,368] - [rest_client:3465] INFO - query params : scan_consistency=request_plus&statement=SELECT+%2A+FROM+default+WHERE++join_yr+%3E+2010+and+join_yr+%3C+2014+ORDER+BY+_id+
[2019-12-10 23:21:22,623] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 220.930258ms
[2019-12-10 23:21:22,679] - [tuq_helper:214] INFO -  Analyzing Actual Result
[2019-12-10 23:21:22,680] - [tuq_helper:216] INFO -  Analyzing Expected Result
[2019-12-10 23:21:23,989] - [basetestcase:2436] INFO - list of n1ql nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:21:23,989] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee33634bff2658421884081898830f463cjoin_yr'
[2019-12-10 23:21:23,996] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee33634bff2658421884081898830f463cjoin_yr%27
[2019-12-10 23:21:24,007] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 7.67865ms
[2019-12-10 23:21:24,038] - [basetestcase:2436] INFO - list of index nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:21:24,043] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:21:24,043] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:21:25,149] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9000: True content: "/opt/build/ns_server/logs/n_0" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))).
[2019-12-10 23:21:25,150] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: zgrep "panic" "/opt/build/ns_server/logs/n_0"/indexer.log* | wc -l
[2019-12-10 23:21:25,298] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:21:25,315] - [basetestcase:2436] INFO - list of kv nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:21:25,319] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:21:25,319] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:21:26,329] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9000: True content: "/opt/build/ns_server/logs/n_0" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))).
[2019-12-10 23:21:26,330] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: zgrep "panic" "/opt/build/ns_server/logs/n_0"/projector.log* | wc -l
[2019-12-10 23:21:26,478] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:21:26,498] - [basetestcase:2436] INFO - list of n1ql nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:21:26,498] - [tuq_helper:516] INFO - CHECK FOR PRIMARY INDEXES
[2019-12-10 23:21:26,499] - [tuq_helper:523] INFO - DROP PRIMARY INDEX ON default USING GSI
[2019-12-10 23:21:26,499] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2019-12-10 23:21:26,506] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2019-12-10 23:21:26,513] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 5.055971ms
[2019-12-10 23:21:26,514] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: rm -f /opt/couchbase/var/lib/couchbase/meta;rm -f /tmp/log_upr_client.sock
[2019-12-10 23:21:26,655] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:21:26,656] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: killall cbq-engine
[2019-12-10 23:21:26,803] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:21:26,805] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: killall tuqtng
[2019-12-10 23:21:26,989] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:21:26,990] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: killall indexer
[2019-12-10 23:21:27,165] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:21:27,189] - [basetestcase:397] INFO - ------- Cluster statistics -------
[2019-12-10 23:21:27,189] - [basetestcase:399] INFO - 127.0.0.1:9000 => {'swap_mem_used': 0, 'cpu_utilization': 28.89518413597734, 'mem_free': 13314023424, 'swap_mem_total': 0, 'mem_total': 15466930176, 'services': [u'index', u'kv', u'n1ql']}
[2019-12-10 23:21:27,190] - [basetestcase:400] INFO - --- End of cluster statistics ---
[2019-12-10 23:21:27,190] - [basetestcase:462] INFO - ==============  basetestcase cleanup was started for test #12 test_remove_bucket_and_query ==============
[2019-12-10 23:21:27,310] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9000
[2019-12-10 23:21:27,327] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9000 is running
[2019-12-10 23:21:27,334] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9001
[2019-12-10 23:21:27,341] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9001 is running
[2019-12-10 23:21:27,353] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9002
[2019-12-10 23:21:27,361] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9002 is running
[2019-12-10 23:21:27,370] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9003
[2019-12-10 23:21:27,380] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9003 is running
[2019-12-10 23:21:27,381] - [basetestcase:480] INFO - ==============  basetestcase cleanup was finished for test #12 test_remove_bucket_and_query ==============
ok

----------------------------------------------------------------------
Ran 1 test in 36.864s

OK
test_change_bucket_properties (2i.cluster_ops_2i.SecondaryIndexingClusterOpsTests) ... Cluster instance shutdown with force
summary so far suite 2i.indexscans_2i.SecondaryIndexingScanTests , pass 9 , fail 0
summary so far suite 2i.indexcreatedrop_2i.SecondaryIndexingCreateDropTests , pass 2 , fail 0
summary so far suite 2i.cluster_ops_2i.SecondaryIndexingClusterOpsTests , pass 1 , fail 0
testrunner logs, diags and results are available under /opt/build/testrunner/logs/testrunner-19-Dec-10_23-14-02/test_12
Logs will be stored at /opt/build/testrunner/logs/testrunner-19-Dec-10_23-14-02/test_13

./testrunner -i b/resources/dev-4-nodes-xdcr_n1ql_2i.ini -p makefile=True -t 2i.cluster_ops_2i.SecondaryIndexingClusterOpsTests.test_change_bucket_properties,groups=simple:and:no_orderby_groupby:range,dataset=default,doc-per-day=1,use_gsi_for_primary=True,use_gsi_for_secondary=True,GROUP=gsi

Test Input params:
{'doc-per-day': '1', 'use_gsi_for_primary': 'True', 'GROUP': 'gsi', 'use_gsi_for_secondary': 'True', 'gsi_type': 'forestdb', 'makefile': 'True', 'conf_file': 'conf/simple_2i_n1ql.conf', 'dataset': 'default', 'cluster_name': 'dev-4-nodes-xdcr_n1ql_2i', 'ini': 'b/resources/dev-4-nodes-xdcr_n1ql_2i.ini', 'groups': 'simple:and:no_orderby_groupby:range', 'case_number': 13, 'num_nodes': 4, 'logs_folder': '/opt/build/testrunner/logs/testrunner-19-Dec-10_23-14-02/test_13', 'spec': 'simple_2i_n1ql'}
[2019-12-10 23:21:27,485] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:21:27,485] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:21:28,580] - [rest_client:2127] INFO - Node version in cluster 6.5.0-4928-rel-enterprise
[2019-12-10 23:21:28,595] - [rest_client:2137] INFO - Node versions in cluster [u'6.5.0-4928-rel-enterprise']
[2019-12-10 23:21:28,595] - [basetestcase:218] INFO - ==============  basetestcase setup was started for test #13 test_change_bucket_properties==============
[2019-12-10 23:21:28,687] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9000
[2019-12-10 23:21:28,694] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9000 is running
[2019-12-10 23:21:28,700] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9001
[2019-12-10 23:21:28,706] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9001 is running
[2019-12-10 23:21:28,713] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9002
[2019-12-10 23:21:28,720] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9002 is running
[2019-12-10 23:21:28,727] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9003
[2019-12-10 23:21:28,733] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9003 is running
[2019-12-10 23:21:28,734] - [basetestcase:265] INFO - initializing cluster
[2019-12-10 23:21:29,522] - [task:132] INFO - server: ip:127.0.0.1 port:9000 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_0@cb.local', 'uptime': u'439', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9000', 'storage': [], 'moxi': 11211, 'port': u'9000', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12000, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'index', u'kv', u'n1ql'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13314023424, 'memoryTotal': 15466930176, 'memoryQuota': 7650, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:21:29,530] - [task:171] INFO - quota for index service will be 256 MB
[2019-12-10 23:21:29,531] - [task:173] INFO - set index quota to node 127.0.0.1 
[2019-12-10 23:21:29,531] - [rest_client:965] INFO - pools/default params : indexMemoryQuota=256
[2019-12-10 23:21:29,547] - [rest_client:953] INFO - pools/default params : memoryQuota=7650
[2019-12-10 23:21:29,557] - [rest_client:931] INFO - /node/controller/setupServices params on 127.0.0.1:9000:services=kv%2Cindex%2Cn1ql&password=asdasd&user=Administrator&hostname=127.0.0.1%3A9000
[2019-12-10 23:21:29,564] - [rest_client:840] ERROR - POST http://127.0.0.1:9000//node/controller/setupServices body: services=kv%2Cindex%2Cn1ql&password=asdasd&user=Administrator&hostname=127.0.0.1%3A9000 headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown ["cannot change node services after cluster is provisioned"] auth: Administrator:asdasd
[2019-12-10 23:21:29,565] - [rest_client:936] INFO - This node is already provisioned with services, we do not consider this as failure for test case
[2019-12-10 23:21:29,565] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9000:username=Administrator&password=asdasd&port=9000
[2019-12-10 23:21:29,637] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:21:29,638] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:21:30,623] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9000/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:21:30,779] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:21:30,787] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9000: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:21:30,787] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:21:30,793] - [rest_client:840] ERROR - POST http://127.0.0.1:9000/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:21:30,821] - [task:132] INFO - server: ip:127.0.0.1 port:9001 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_1@cb.local', 'uptime': u'444', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9001', 'storage': [], 'moxi': 11211, 'port': u'9001', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12002, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13399191552, 'memoryTotal': 15466930176, 'memoryQuota': 7906, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:21:30,831] - [rest_client:953] INFO - pools/default params : memoryQuota=7906
[2019-12-10 23:21:30,839] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9001:username=Administrator&password=asdasd&port=9001
[2019-12-10 23:21:30,898] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:21:30,898] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:21:31,874] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9001/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:21:32,029] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:21:32,036] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9001: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:21:32,036] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:21:32,042] - [rest_client:840] ERROR - POST http://127.0.0.1:9001/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:21:32,069] - [task:132] INFO - server: ip:127.0.0.1 port:9002 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_2@cb.local', 'uptime': u'444', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9002', 'storage': [], 'moxi': 11211, 'port': u'9002', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12004, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13399191552, 'memoryTotal': 15466930176, 'memoryQuota': 7906, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:21:32,079] - [rest_client:953] INFO - pools/default params : memoryQuota=7906
[2019-12-10 23:21:32,087] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9002:username=Administrator&password=asdasd&port=9002
[2019-12-10 23:21:32,142] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:21:32,143] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:21:33,131] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9002/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:21:33,283] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:21:33,293] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9002: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:21:33,294] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:21:33,299] - [rest_client:840] ERROR - POST http://127.0.0.1:9002/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:21:33,326] - [task:132] INFO - server: ip:127.0.0.1 port:9003 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_3@cb.local', 'uptime': u'444', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9003', 'storage': [], 'moxi': 11211, 'port': u'9003', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12006, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13397159936, 'memoryTotal': 15466930176, 'memoryQuota': 7906, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:21:33,335] - [rest_client:953] INFO - pools/default params : memoryQuota=7906
[2019-12-10 23:21:33,343] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9003:username=Administrator&password=asdasd&port=9003
[2019-12-10 23:21:33,405] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:21:33,406] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:21:34,383] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9003/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:21:34,535] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:21:34,543] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9003: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:21:34,543] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:21:34,548] - [rest_client:840] ERROR - POST http://127.0.0.1:9003/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:21:34,580] - [basetestcase:2218] INFO - **** add built-in 'cbadminbucket' user to node 127.0.0.1 ****
[2019-12-10 23:21:34,684] - [basetestcase:530] INFO - sleep for 5 secs.  ...
[2019-12-10 23:21:39,690] - [basetestcase:2223] INFO - **** add 'admin' role to 'cbadminbucket' user ****
[2019-12-10 23:21:39,714] - [basetestcase:291] INFO - done initializing cluster
[2019-12-10 23:21:39,720] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:21:39,721] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:21:40,697] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9000/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:21:40,854] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:21:40,855] - [basetestcase:2770] INFO - Enabled diag/eval for non-local hosts from 127.0.0.1
[2019-12-10 23:21:41,678] - [rest_client:2430] INFO - http://127.0.0.1:9000/pools/default/buckets with param: bucketType=membase&evictionPolicy=valueOnly&threadsNumber=3&ramQuotaMB=7650&authType=sasl&name=default&flushEnabled=1&replicaNumber=1&compressionMode=passive&saslPassword=None&replicaIndex=1
[2019-12-10 23:21:41,706] - [rest_client:2455] INFO - 0.03 seconds to create bucket default
[2019-12-10 23:21:41,710] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:21:41,850] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:21:41,935] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:21:41,978] - [task:351] WARNING - vbucket map not ready after try 0
[2019-12-10 23:21:41,978] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:21:42,064] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:21:42,142] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:21:42,183] - [task:351] WARNING - vbucket map not ready after try 1
[2019-12-10 23:21:42,184] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:21:42,463] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:21:42,546] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:21:42,589] - [task:351] WARNING - vbucket map not ready after try 2
[2019-12-10 23:21:42,589] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:21:42,665] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:21:42,743] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:21:42,796] - [task:351] WARNING - vbucket map not ready after try 3
[2019-12-10 23:21:42,797] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:21:43,008] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:21:43,086] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:21:43,131] - [task:351] WARNING - vbucket map not ready after try 4
[2019-12-10 23:21:43,131] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:21:43,208] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:21:43,286] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:21:43,326] - [task:351] WARNING - vbucket map not ready after try 5
[2019-12-10 23:21:43,330] - [basetestcase:383] INFO - ==============  basetestcase setup was finished for test #13 test_change_bucket_properties ==============
[2019-12-10 23:21:43,354] - [basetestcase:397] INFO - ------- Cluster statistics -------
[2019-12-10 23:21:43,354] - [basetestcase:399] INFO - 127.0.0.1:9000 => {'swap_mem_used': 0, 'cpu_utilization': 12.21374045801527, 'mem_free': 13409021952, 'swap_mem_total': 0, 'mem_total': 15466930176, 'services': [u'index', u'kv', u'n1ql']}
[2019-12-10 23:21:43,354] - [basetestcase:400] INFO - --- End of cluster statistics ---
[2019-12-10 23:21:43,366] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is warmup
[2019-12-10 23:21:43,367] - [newtuq:22] INFO - Waiting for cluster to become healthy
[2019-12-10 23:21:43,367] - [basetestcase:530] INFO - sleep for 5 secs.  ...
[2019-12-10 23:21:48,380] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:21:48,393] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is healthy
[2019-12-10 23:21:48,393] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:21:48,406] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is healthy
[2019-12-10 23:21:48,407] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:21:48,420] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is healthy
[2019-12-10 23:21:48,421] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:21:48,433] - [basetestcase:2436] INFO - list of index nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:21:48,450] - [rest_client:1706] INFO - {'indexer.settings.storage_mode': 'forestdb'} set
[2019-12-10 23:21:48,493] - [rest_client:1706] INFO - {'indexer.api.enableTestServer': True} set
[2019-12-10 23:21:48,497] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:21:48,497] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:21:50,879] - [basetestcase:2545] INFO - create 2016 to default documents...
[2019-12-10 23:21:50,934] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:21:52,654] - [basetestcase:2557] INFO - LOAD IS FINISHED
[2019-12-10 23:21:52,698] - [basetestcase:2436] INFO - list of n1ql nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:21:52,699] - [newtuq:83] INFO - ip:127.0.0.1 port:9000 ssh_username:Administrator
[2019-12-10 23:21:52,699] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2019-12-10 23:21:52,707] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2019-12-10 23:21:52,801] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 90.990668ms
[2019-12-10 23:21:52,801] - [tuq_helper:147] INFO - RUN QUERY CREATE PRIMARY INDEX ON default  USING GSI
[2019-12-10 23:21:52,810] - [rest_client:3465] INFO - query params : statement=CREATE+PRIMARY+INDEX+ON+default++USING+GSI
[2019-12-10 23:21:53,530] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 716.847942ms
[2019-12-10 23:21:53,530] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2019-12-10 23:21:53,541] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2019-12-10 23:21:53,556] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 11.909172ms
[2019-12-10 23:21:53,603] - [basetestcase:2436] INFO - list of n1ql nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:21:54,585] - [tuq_helper:147] INFO - RUN QUERY CREATE INDEX employeee073071c81c34771a76c4a32fd75baadjoin_yr ON default(join_yr) USING GSI  WITH {'defer_build': True}
[2019-12-10 23:21:54,593] - [rest_client:3465] INFO - query params : statement=CREATE+INDEX+employeee073071c81c34771a76c4a32fd75baadjoin_yr+ON+default%28join_yr%29+USING+GSI++WITH+%7B%27defer_build%27%3A+True%7D
[2019-12-10 23:21:54,866] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 269.970881ms
[2019-12-10 23:21:54,867] - [base_2i:109] INFO - BUILD INDEX on default(employeee073071c81c34771a76c4a32fd75baadjoin_yr) USING GSI
[2019-12-10 23:21:55,868] - [tuq_helper:147] INFO - RUN QUERY BUILD INDEX on default(employeee073071c81c34771a76c4a32fd75baadjoin_yr) USING GSI
[2019-12-10 23:21:55,876] - [rest_client:3465] INFO - query params : statement=BUILD+INDEX+on+default%28employeee073071c81c34771a76c4a32fd75baadjoin_yr%29+USING+GSI
[2019-12-10 23:21:55,953] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 73.573077ms
[2019-12-10 23:21:55,953] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employeee073071c81c34771a76c4a32fd75baadjoin_yr'
[2019-12-10 23:21:55,963] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employeee073071c81c34771a76c4a32fd75baadjoin_yr%27
[2019-12-10 23:21:55,983] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 16.491222ms
[2019-12-10 23:21:56,985] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employeee073071c81c34771a76c4a32fd75baadjoin_yr'
[2019-12-10 23:21:56,992] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employeee073071c81c34771a76c4a32fd75baadjoin_yr%27
[2019-12-10 23:21:57,009] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 13.550843ms
[2019-12-10 23:21:58,010] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employeee073071c81c34771a76c4a32fd75baadjoin_yr'
[2019-12-10 23:21:58,018] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employeee073071c81c34771a76c4a32fd75baadjoin_yr%27
[2019-12-10 23:21:58,035] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 14.115537ms
[2019-12-10 23:21:59,036] - [tuq_helper:147] INFO - RUN QUERY EXPLAIN SELECT * FROM default WHERE  join_yr > 2010 and join_yr < 2014 ORDER BY _id 
[2019-12-10 23:21:59,044] - [rest_client:3465] INFO - query params : statement=EXPLAIN+SELECT+%2A+FROM+default+WHERE++join_yr+%3E+2010+and+join_yr+%3C+2014+ORDER+BY+_id+
[2019-12-10 23:21:59,052] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 4.921541ms
[2019-12-10 23:21:59,052] - [base_2i:261] INFO - {u'status': u'success', u'metrics': {u'elapsedTime': u'4.921541ms', u'executionTime': u'4.866699ms', u'resultSize': 895, u'resultCount': 1}, u'results': [{u'text': u'SELECT * FROM default WHERE  join_yr > 2010 and join_yr < 2014 ORDER BY _id', u'plan': {u'#operator': u'Sequence', u'~children': [{u'#operator': u'Sequence', u'~children': [{u'index': u'employeee073071c81c34771a76c4a32fd75baadjoin_yr', u'namespace': u'default', u'keyspace': u'default', u'#operator': u'IndexScan3', u'index_projection': {u'primary_key': True}, u'spans': [{u'range': [{u'high': u'2014', u'low': u'2010', u'inclusion': 0}], u'exact': True}], u'using': u'gsi', u'index_id': u'746ddb2d1bcbad07'}, {u'keyspace': u'default', u'#operator': u'Fetch', u'namespace': u'default'}, {u'#operator': u'Parallel', u'~child': {u'#operator': u'Sequence', u'~children': [{u'#operator': u'Filter', u'condition': u'((2010 < (`default`.`join_yr`)) and ((`default`.`join_yr`) < 2014))'}, {u'#operator': u'InitialProject', u'result_terms': [{u'expr': u'self', u'star': True}]}]}}]}, {u'#operator': u'Order', u'sort_terms': [{u'expr': u'(`default`.`_id`)'}]}, {u'#operator': u'FinalProject'}]}}], u'requestID': u'8e08d1b9-68aa-40f3-aadc-f19199dd67c1', u'signature': u'json'}
[2019-12-10 23:21:59,053] - [tuq_generators:70] INFO - FROM clause ===== is default
[2019-12-10 23:21:59,054] - [tuq_generators:72] INFO - WHERE clause ===== is   doc["join_yr"] > 2010 and doc["join_yr"] < 2014 
[2019-12-10 23:21:59,054] - [tuq_generators:74] INFO - UNNEST clause ===== is None
[2019-12-10 23:21:59,054] - [tuq_generators:76] INFO - SELECT clause ===== is {"*" : doc,}
[2019-12-10 23:21:59,054] - [tuq_generators:417] INFO - ORDER clause ========= is doc["_id"],
[2019-12-10 23:21:59,055] - [tuq_generators:417] INFO - ORDER clause ========= is doc["_id"],
[2019-12-10 23:21:59,154] - [tuq_generators:417] INFO - ORDER clause ========= is doc["_id"],
[2019-12-10 23:21:59,172] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM default WHERE  join_yr > 2010 and join_yr < 2014 ORDER BY _id 
[2019-12-10 23:21:59,180] - [rest_client:3465] INFO - query params : scan_consistency=request_plus&statement=SELECT+%2A+FROM+default+WHERE++join_yr+%3E+2010+and+join_yr+%3C+2014+ORDER+BY+_id+
[2019-12-10 23:21:59,430] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 219.288336ms
[2019-12-10 23:21:59,493] - [tuq_helper:214] INFO -  Analyzing Actual Result
[2019-12-10 23:21:59,494] - [tuq_helper:216] INFO -  Analyzing Expected Result
[2019-12-10 23:21:59,511] - [rest_client:2499] INFO - http://127.0.0.1:9000/pools/default/buckets/default with param: 
[2019-12-10 23:21:59,524] - [rest_client:2507] INFO - bucket default updated
[2019-12-10 23:21:59,525] - [tuq_helper:147] INFO - RUN QUERY EXPLAIN SELECT * FROM default WHERE  join_yr > 2010 and join_yr < 2014 ORDER BY _id 
[2019-12-10 23:21:59,533] - [rest_client:3465] INFO - query params : statement=EXPLAIN+SELECT+%2A+FROM+default+WHERE++join_yr+%3E+2010+and+join_yr+%3C+2014+ORDER+BY+_id+
[2019-12-10 23:21:59,539] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 2.636343ms
[2019-12-10 23:21:59,539] - [base_2i:261] INFO - {u'status': u'success', u'metrics': {u'elapsedTime': u'2.636343ms', u'executionTime': u'2.491606ms', u'resultSize': 895, u'resultCount': 1}, u'results': [{u'text': u'SELECT * FROM default WHERE  join_yr > 2010 and join_yr < 2014 ORDER BY _id', u'plan': {u'#operator': u'Sequence', u'~children': [{u'#operator': u'Sequence', u'~children': [{u'index': u'employeee073071c81c34771a76c4a32fd75baadjoin_yr', u'namespace': u'default', u'keyspace': u'default', u'#operator': u'IndexScan3', u'index_projection': {u'primary_key': True}, u'spans': [{u'range': [{u'high': u'2014', u'low': u'2010', u'inclusion': 0}], u'exact': True}], u'using': u'gsi', u'index_id': u'746ddb2d1bcbad07'}, {u'keyspace': u'default', u'#operator': u'Fetch', u'namespace': u'default'}, {u'#operator': u'Parallel', u'~child': {u'#operator': u'Sequence', u'~children': [{u'#operator': u'Filter', u'condition': u'((2010 < (`default`.`join_yr`)) and ((`default`.`join_yr`) < 2014))'}, {u'#operator': u'InitialProject', u'result_terms': [{u'expr': u'self', u'star': True}]}]}}]}, {u'#operator': u'Order', u'sort_terms': [{u'expr': u'(`default`.`_id`)'}]}, {u'#operator': u'FinalProject'}]}}], u'requestID': u'6fa8f719-1163-4816-b080-c0baf76b0c0b', u'signature': u'json'}
[2019-12-10 23:21:59,540] - [tuq_generators:70] INFO - FROM clause ===== is default
[2019-12-10 23:21:59,540] - [tuq_generators:72] INFO - WHERE clause ===== is   doc["join_yr"] > 2010 and doc["join_yr"] < 2014 
[2019-12-10 23:21:59,540] - [tuq_generators:74] INFO - UNNEST clause ===== is None
[2019-12-10 23:21:59,541] - [tuq_generators:76] INFO - SELECT clause ===== is {"*" : doc,}
[2019-12-10 23:21:59,541] - [tuq_generators:417] INFO - ORDER clause ========= is doc["_id"],
[2019-12-10 23:21:59,541] - [tuq_generators:417] INFO - ORDER clause ========= is doc["_id"],
[2019-12-10 23:21:59,604] - [tuq_generators:417] INFO - ORDER clause ========= is doc["_id"],
[2019-12-10 23:21:59,618] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM default WHERE  join_yr > 2010 and join_yr < 2014 ORDER BY _id 
[2019-12-10 23:21:59,626] - [rest_client:3465] INFO - query params : scan_consistency=request_plus&statement=SELECT+%2A+FROM+default+WHERE++join_yr+%3E+2010+and+join_yr+%3C+2014+ORDER+BY+_id+
[2019-12-10 23:21:59,859] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 207.942319ms
[2019-12-10 23:21:59,919] - [tuq_helper:214] INFO -  Analyzing Actual Result
[2019-12-10 23:21:59,920] - [tuq_helper:216] INFO -  Analyzing Expected Result
[2019-12-10 23:21:59,929] - [tuq_helper:147] INFO - RUN QUERY DROP INDEX default.employeee073071c81c34771a76c4a32fd75baadjoin_yr USING GSI
[2019-12-10 23:21:59,938] - [rest_client:3465] INFO - query params : statement=DROP+INDEX+default.employeee073071c81c34771a76c4a32fd75baadjoin_yr+USING+GSI
[2019-12-10 23:22:00,037] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 96.701836ms
[2019-12-10 23:22:00,038] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employeee073071c81c34771a76c4a32fd75baadjoin_yr'
[2019-12-10 23:22:00,048] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employeee073071c81c34771a76c4a32fd75baadjoin_yr%27
[2019-12-10 23:22:00,075] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 21.328635ms
[2019-12-10 23:22:00,105] - [basetestcase:2436] INFO - list of index nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:22:00,109] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:22:00,109] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:22:01,238] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9000: True content: "/opt/build/ns_server/logs/n_0" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))).
[2019-12-10 23:22:01,240] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: zgrep "panic" "/opt/build/ns_server/logs/n_0"/indexer.log* | wc -l
[2019-12-10 23:22:01,403] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:22:01,422] - [basetestcase:2436] INFO - list of kv nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:22:01,426] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:22:01,426] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:22:02,536] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9000: True content: "/opt/build/ns_server/logs/n_0" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))).
[2019-12-10 23:22:02,537] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: zgrep "panic" "/opt/build/ns_server/logs/n_0"/projector.log* | wc -l
[2019-12-10 23:22:02,701] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:22:02,721] - [basetestcase:2436] INFO - list of n1ql nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:22:02,722] - [tuq_helper:516] INFO - CHECK FOR PRIMARY INDEXES
[2019-12-10 23:22:02,722] - [tuq_helper:523] INFO - DROP PRIMARY INDEX ON default USING GSI
[2019-12-10 23:22:02,722] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2019-12-10 23:22:02,730] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2019-12-10 23:22:02,749] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 16.705975ms
[2019-12-10 23:22:02,749] - [tuq_helper:147] INFO - RUN QUERY DROP PRIMARY INDEX ON default USING GSI
[2019-12-10 23:22:02,757] - [rest_client:3465] INFO - query params : statement=DROP+PRIMARY+INDEX+ON+default+USING+GSI
[2019-12-10 23:22:02,836] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 76.068121ms
[2019-12-10 23:22:02,836] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: rm -f /opt/couchbase/var/lib/couchbase/meta;rm -f /tmp/log_upr_client.sock
[2019-12-10 23:22:02,990] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:22:02,991] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: killall cbq-engine
[2019-12-10 23:22:03,138] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:22:03,139] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: killall tuqtng
[2019-12-10 23:22:03,329] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:22:03,331] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: killall indexer
[2019-12-10 23:22:03,520] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:22:03,546] - [basetestcase:397] INFO - ------- Cluster statistics -------
[2019-12-10 23:22:03,546] - [basetestcase:399] INFO - 127.0.0.1:9000 => {'swap_mem_used': 0, 'cpu_utilization': 26.68539325842697, 'mem_free': 13302468608, 'swap_mem_total': 0, 'mem_total': 15466930176, 'services': [u'index', u'kv', u'n1ql']}
[2019-12-10 23:22:03,547] - [basetestcase:400] INFO - --- End of cluster statistics ---
[2019-12-10 23:22:03,547] - [basetestcase:462] INFO - ==============  basetestcase cleanup was started for test #13 test_change_bucket_properties ==============
[2019-12-10 23:22:03,601] - [bucket_helper:143] INFO - deleting existing buckets [u'default'] on 127.0.0.1
[2019-12-10 23:22:03,601] - [bucket_helper:145] INFO - remove bucket default ...
[2019-12-10 23:22:04,040] - [bucket_helper:159] INFO - deleted bucket : default from 127.0.0.1
[2019-12-10 23:22:04,041] - [bucket_helper:237] INFO - waiting for bucket deletion to complete....
[2019-12-10 23:22:04,047] - [rest_client:133] INFO - node 127.0.0.1 existing buckets : []
[2019-12-10 23:22:04,047] - [bucket_helper:169] INFO - sleep 2 seconds to make sure all buckets ([u'default']) were deleted completely.
[2019-12-10 23:22:06,117] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9000
[2019-12-10 23:22:06,123] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9000 is running
[2019-12-10 23:22:06,130] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9001
[2019-12-10 23:22:06,138] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9001 is running
[2019-12-10 23:22:06,146] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9002
[2019-12-10 23:22:06,153] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9002 is running
[2019-12-10 23:22:06,159] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9003
[2019-12-10 23:22:06,166] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9003 is running
[2019-12-10 23:22:06,166] - [basetestcase:480] INFO - ==============  basetestcase cleanup was finished for test #13 test_change_bucket_properties ==============
ok

----------------------------------------------------------------------
Ran 1 test in 38.703s

OK
test_flush_bucket_and_query (2i.cluster_ops_2i.SecondaryIndexingClusterOpsTests) ... Cluster instance shutdown with force
summary so far suite 2i.indexscans_2i.SecondaryIndexingScanTests , pass 9 , fail 0
summary so far suite 2i.indexcreatedrop_2i.SecondaryIndexingCreateDropTests , pass 2 , fail 0
summary so far suite 2i.cluster_ops_2i.SecondaryIndexingClusterOpsTests , pass 2 , fail 0
testrunner logs, diags and results are available under /opt/build/testrunner/logs/testrunner-19-Dec-10_23-14-02/test_13
Logs will be stored at /opt/build/testrunner/logs/testrunner-19-Dec-10_23-14-02/test_14

./testrunner -i b/resources/dev-4-nodes-xdcr_n1ql_2i.ini -p makefile=True -t 2i.cluster_ops_2i.SecondaryIndexingClusterOpsTests.test_flush_bucket_and_query,groups=simple:and:no_orderby_groupby:range,dataset=default,doc-per-day=1,use_gsi_for_primary=True,use_gsi_for_secondary=True,GROUP=gsi

Test Input params:
{'doc-per-day': '1', 'use_gsi_for_primary': 'True', 'GROUP': 'gsi', 'use_gsi_for_secondary': 'True', 'gsi_type': 'forestdb', 'makefile': 'True', 'conf_file': 'conf/simple_2i_n1ql.conf', 'dataset': 'default', 'cluster_name': 'dev-4-nodes-xdcr_n1ql_2i', 'ini': 'b/resources/dev-4-nodes-xdcr_n1ql_2i.ini', 'groups': 'simple:and:no_orderby_groupby:range', 'case_number': 14, 'num_nodes': 4, 'logs_folder': '/opt/build/testrunner/logs/testrunner-19-Dec-10_23-14-02/test_14', 'spec': 'simple_2i_n1ql'}
[2019-12-10 23:22:06,227] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:22:06,227] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:22:07,271] - [rest_client:2127] INFO - Node version in cluster 6.5.0-4928-rel-enterprise
[2019-12-10 23:22:07,285] - [rest_client:2137] INFO - Node versions in cluster [u'6.5.0-4928-rel-enterprise']
[2019-12-10 23:22:07,286] - [basetestcase:218] INFO - ==============  basetestcase setup was started for test #14 test_flush_bucket_and_query==============
[2019-12-10 23:22:07,379] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9000
[2019-12-10 23:22:07,385] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9000 is running
[2019-12-10 23:22:07,391] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9001
[2019-12-10 23:22:07,398] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9001 is running
[2019-12-10 23:22:07,404] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9002
[2019-12-10 23:22:07,411] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9002 is running
[2019-12-10 23:22:07,418] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9003
[2019-12-10 23:22:07,424] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9003 is running
[2019-12-10 23:22:07,426] - [basetestcase:265] INFO - initializing cluster
[2019-12-10 23:22:08,238] - [task:132] INFO - server: ip:127.0.0.1 port:9000 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_0@cb.local', 'uptime': u'479', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9000', 'storage': [], 'moxi': 11211, 'port': u'9000', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12000, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'index', u'kv', u'n1ql'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13340114944, 'memoryTotal': 15466930176, 'memoryQuota': 7650, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:22:08,246] - [task:171] INFO - quota for index service will be 256 MB
[2019-12-10 23:22:08,246] - [task:173] INFO - set index quota to node 127.0.0.1 
[2019-12-10 23:22:08,247] - [rest_client:965] INFO - pools/default params : indexMemoryQuota=256
[2019-12-10 23:22:08,271] - [rest_client:953] INFO - pools/default params : memoryQuota=7650
[2019-12-10 23:22:08,287] - [rest_client:931] INFO - /node/controller/setupServices params on 127.0.0.1:9000:services=kv%2Cindex%2Cn1ql&password=asdasd&user=Administrator&hostname=127.0.0.1%3A9000
[2019-12-10 23:22:08,294] - [rest_client:840] ERROR - POST http://127.0.0.1:9000//node/controller/setupServices body: services=kv%2Cindex%2Cn1ql&password=asdasd&user=Administrator&hostname=127.0.0.1%3A9000 headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown ["cannot change node services after cluster is provisioned"] auth: Administrator:asdasd
[2019-12-10 23:22:08,295] - [rest_client:936] INFO - This node is already provisioned with services, we do not consider this as failure for test case
[2019-12-10 23:22:08,295] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9000:username=Administrator&password=asdasd&port=9000
[2019-12-10 23:22:08,355] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:22:08,356] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:22:09,357] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9000/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:22:09,514] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:22:09,526] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9000: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:22:09,526] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:22:09,532] - [rest_client:840] ERROR - POST http://127.0.0.1:9000/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:22:09,559] - [task:132] INFO - server: ip:127.0.0.1 port:9001 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_1@cb.local', 'uptime': u'479', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9001', 'storage': [], 'moxi': 11211, 'port': u'9001', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12002, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13340127232, 'memoryTotal': 15466930176, 'memoryQuota': 7906, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:22:09,569] - [rest_client:953] INFO - pools/default params : memoryQuota=7906
[2019-12-10 23:22:09,578] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9001:username=Administrator&password=asdasd&port=9001
[2019-12-10 23:22:09,633] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:22:09,633] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:22:10,650] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9001/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:22:10,811] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:22:10,821] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9001: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:22:10,822] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:22:10,828] - [rest_client:840] ERROR - POST http://127.0.0.1:9001/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:22:10,856] - [task:132] INFO - server: ip:127.0.0.1 port:9002 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_2@cb.local', 'uptime': u'484', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9002', 'storage': [], 'moxi': 11211, 'port': u'9002', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12004, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13385355264, 'memoryTotal': 15466930176, 'memoryQuota': 7906, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:22:10,866] - [rest_client:953] INFO - pools/default params : memoryQuota=7906
[2019-12-10 23:22:10,875] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9002:username=Administrator&password=asdasd&port=9002
[2019-12-10 23:22:10,932] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:22:10,932] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:22:11,941] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9002/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:22:12,098] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:22:12,105] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9002: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:22:12,106] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:22:12,115] - [rest_client:840] ERROR - POST http://127.0.0.1:9002/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:22:12,142] - [task:132] INFO - server: ip:127.0.0.1 port:9003 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_3@cb.local', 'uptime': u'484', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9003', 'storage': [], 'moxi': 11211, 'port': u'9003', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12006, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13384593408, 'memoryTotal': 15466930176, 'memoryQuota': 7906, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:22:12,152] - [rest_client:953] INFO - pools/default params : memoryQuota=7906
[2019-12-10 23:22:12,160] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9003:username=Administrator&password=asdasd&port=9003
[2019-12-10 23:22:12,219] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:22:12,219] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:22:13,243] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9003/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:22:13,401] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:22:13,409] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9003: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:22:13,409] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:22:13,415] - [rest_client:840] ERROR - POST http://127.0.0.1:9003/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:22:13,446] - [basetestcase:2218] INFO - **** add built-in 'cbadminbucket' user to node 127.0.0.1 ****
[2019-12-10 23:22:13,555] - [basetestcase:530] INFO - sleep for 5 secs.  ...
[2019-12-10 23:22:18,560] - [basetestcase:2223] INFO - **** add 'admin' role to 'cbadminbucket' user ****
[2019-12-10 23:22:18,582] - [basetestcase:291] INFO - done initializing cluster
[2019-12-10 23:22:18,586] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:22:18,586] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:22:19,580] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9000/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:22:19,738] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:22:19,740] - [basetestcase:2770] INFO - Enabled diag/eval for non-local hosts from 127.0.0.1
[2019-12-10 23:22:20,564] - [rest_client:2430] INFO - http://127.0.0.1:9000/pools/default/buckets with param: bucketType=membase&evictionPolicy=valueOnly&threadsNumber=3&ramQuotaMB=7650&authType=sasl&name=default&flushEnabled=1&replicaNumber=1&compressionMode=passive&saslPassword=None&replicaIndex=1
[2019-12-10 23:22:20,583] - [rest_client:2455] INFO - 0.02 seconds to create bucket default
[2019-12-10 23:22:20,583] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:22:20,745] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:22:21,050] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:22:21,095] - [task:351] WARNING - vbucket map not ready after try 0
[2019-12-10 23:22:21,095] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:22:21,174] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:22:21,255] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:22:21,299] - [task:351] WARNING - vbucket map not ready after try 1
[2019-12-10 23:22:21,299] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:22:21,379] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:22:21,586] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:22:21,629] - [task:351] WARNING - vbucket map not ready after try 2
[2019-12-10 23:22:21,630] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:22:21,740] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:22:21,830] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:22:21,873] - [task:351] WARNING - vbucket map not ready after try 3
[2019-12-10 23:22:21,873] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:22:21,950] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:22:22,027] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:22:22,180] - [task:351] WARNING - vbucket map not ready after try 4
[2019-12-10 23:22:22,180] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:22:22,255] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:22:22,332] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:22:22,372] - [task:351] WARNING - vbucket map not ready after try 5
[2019-12-10 23:22:22,418] - [basetestcase:383] INFO - ==============  basetestcase setup was finished for test #14 test_flush_bucket_and_query ==============
[2019-12-10 23:22:22,441] - [basetestcase:397] INFO - ------- Cluster statistics -------
[2019-12-10 23:22:22,441] - [basetestcase:399] INFO - 127.0.0.1:9000 => {'swap_mem_used': 0, 'cpu_utilization': 11.53846153846154, 'mem_free': 13390200832, 'swap_mem_total': 0, 'mem_total': 15466930176, 'services': [u'index', u'kv', u'n1ql']}
[2019-12-10 23:22:22,442] - [basetestcase:400] INFO - --- End of cluster statistics ---
[2019-12-10 23:22:22,453] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is warmup
[2019-12-10 23:22:22,454] - [newtuq:22] INFO - Waiting for cluster to become healthy
[2019-12-10 23:22:22,454] - [basetestcase:530] INFO - sleep for 5 secs.  ...
[2019-12-10 23:22:27,463] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:22:27,477] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is healthy
[2019-12-10 23:22:27,477] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:22:27,494] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is healthy
[2019-12-10 23:22:27,495] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:22:27,508] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is healthy
[2019-12-10 23:22:27,508] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:22:27,521] - [basetestcase:2436] INFO - list of index nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:22:27,539] - [rest_client:1706] INFO - {'indexer.settings.storage_mode': 'forestdb'} set
[2019-12-10 23:22:27,574] - [rest_client:1706] INFO - {'indexer.api.enableTestServer': True} set
[2019-12-10 23:22:27,579] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:22:27,579] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:22:30,002] - [basetestcase:2545] INFO - create 2016 to default documents...
[2019-12-10 23:22:30,054] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:22:31,644] - [basetestcase:2557] INFO - LOAD IS FINISHED
[2019-12-10 23:22:31,682] - [basetestcase:2436] INFO - list of n1ql nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:22:31,682] - [newtuq:83] INFO - ip:127.0.0.1 port:9000 ssh_username:Administrator
[2019-12-10 23:22:31,682] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2019-12-10 23:22:31,689] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2019-12-10 23:22:31,778] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 85.203199ms
[2019-12-10 23:22:31,778] - [tuq_helper:147] INFO - RUN QUERY CREATE PRIMARY INDEX ON default  USING GSI
[2019-12-10 23:22:31,786] - [rest_client:3465] INFO - query params : statement=CREATE+PRIMARY+INDEX+ON+default++USING+GSI
[2019-12-10 23:22:32,573] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 783.027859ms
[2019-12-10 23:22:32,573] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2019-12-10 23:22:32,585] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2019-12-10 23:22:32,599] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 10.632804ms
[2019-12-10 23:22:32,631] - [basetestcase:2436] INFO - list of n1ql nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:22:33,619] - [tuq_helper:147] INFO - RUN QUERY CREATE INDEX employeed352ad8b11d844fbb0962b1bac7b64d7join_yr ON default(join_yr) USING GSI  WITH {'defer_build': True}
[2019-12-10 23:22:33,626] - [rest_client:3465] INFO - query params : statement=CREATE+INDEX+employeed352ad8b11d844fbb0962b1bac7b64d7join_yr+ON+default%28join_yr%29+USING+GSI++WITH+%7B%27defer_build%27%3A+True%7D
[2019-12-10 23:22:33,943] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 312.969344ms
[2019-12-10 23:22:33,943] - [base_2i:109] INFO - BUILD INDEX on default(employeed352ad8b11d844fbb0962b1bac7b64d7join_yr) USING GSI
[2019-12-10 23:22:34,944] - [tuq_helper:147] INFO - RUN QUERY BUILD INDEX on default(employeed352ad8b11d844fbb0962b1bac7b64d7join_yr) USING GSI
[2019-12-10 23:22:34,952] - [rest_client:3465] INFO - query params : statement=BUILD+INDEX+on+default%28employeed352ad8b11d844fbb0962b1bac7b64d7join_yr%29+USING+GSI
[2019-12-10 23:22:35,023] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 67.621339ms
[2019-12-10 23:22:35,024] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employeed352ad8b11d844fbb0962b1bac7b64d7join_yr'
[2019-12-10 23:22:35,033] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employeed352ad8b11d844fbb0962b1bac7b64d7join_yr%27
[2019-12-10 23:22:35,053] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 11.613921ms
[2019-12-10 23:22:36,054] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employeed352ad8b11d844fbb0962b1bac7b64d7join_yr'
[2019-12-10 23:22:36,062] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employeed352ad8b11d844fbb0962b1bac7b64d7join_yr%27
[2019-12-10 23:22:36,076] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 11.328182ms
[2019-12-10 23:22:37,078] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employeed352ad8b11d844fbb0962b1bac7b64d7join_yr'
[2019-12-10 23:22:37,086] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employeed352ad8b11d844fbb0962b1bac7b64d7join_yr%27
[2019-12-10 23:22:37,102] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 13.316377ms
[2019-12-10 23:22:38,103] - [tuq_helper:147] INFO - RUN QUERY EXPLAIN SELECT * FROM default WHERE  join_yr > 2010 and join_yr < 2014 ORDER BY _id 
[2019-12-10 23:22:38,111] - [rest_client:3465] INFO - query params : statement=EXPLAIN+SELECT+%2A+FROM+default+WHERE++join_yr+%3E+2010+and+join_yr+%3C+2014+ORDER+BY+_id+
[2019-12-10 23:22:38,118] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 4.506708ms
[2019-12-10 23:22:38,118] - [base_2i:261] INFO - {u'status': u'success', u'metrics': {u'elapsedTime': u'4.506708ms', u'executionTime': u'4.4612ms', u'resultSize': 895, u'resultCount': 1}, u'results': [{u'text': u'SELECT * FROM default WHERE  join_yr > 2010 and join_yr < 2014 ORDER BY _id', u'plan': {u'#operator': u'Sequence', u'~children': [{u'#operator': u'Sequence', u'~children': [{u'index': u'employeed352ad8b11d844fbb0962b1bac7b64d7join_yr', u'namespace': u'default', u'keyspace': u'default', u'#operator': u'IndexScan3', u'index_projection': {u'primary_key': True}, u'spans': [{u'range': [{u'high': u'2014', u'low': u'2010', u'inclusion': 0}], u'exact': True}], u'using': u'gsi', u'index_id': u'64830753da397643'}, {u'keyspace': u'default', u'#operator': u'Fetch', u'namespace': u'default'}, {u'#operator': u'Parallel', u'~child': {u'#operator': u'Sequence', u'~children': [{u'#operator': u'Filter', u'condition': u'((2010 < (`default`.`join_yr`)) and ((`default`.`join_yr`) < 2014))'}, {u'#operator': u'InitialProject', u'result_terms': [{u'expr': u'self', u'star': True}]}]}}]}, {u'#operator': u'Order', u'sort_terms': [{u'expr': u'(`default`.`_id`)'}]}, {u'#operator': u'FinalProject'}]}}], u'requestID': u'fa130e01-da4d-47e0-986e-8a3aa71aa72e', u'signature': u'json'}
[2019-12-10 23:22:38,119] - [tuq_generators:70] INFO - FROM clause ===== is default
[2019-12-10 23:22:38,119] - [tuq_generators:72] INFO - WHERE clause ===== is   doc["join_yr"] > 2010 and doc["join_yr"] < 2014 
[2019-12-10 23:22:38,119] - [tuq_generators:74] INFO - UNNEST clause ===== is None
[2019-12-10 23:22:38,120] - [tuq_generators:76] INFO - SELECT clause ===== is {"*" : doc,}
[2019-12-10 23:22:38,120] - [tuq_generators:417] INFO - ORDER clause ========= is doc["_id"],
[2019-12-10 23:22:38,120] - [tuq_generators:417] INFO - ORDER clause ========= is doc["_id"],
[2019-12-10 23:22:38,213] - [tuq_generators:417] INFO - ORDER clause ========= is doc["_id"],
[2019-12-10 23:22:38,226] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM default WHERE  join_yr > 2010 and join_yr < 2014 ORDER BY _id 
[2019-12-10 23:22:38,233] - [rest_client:3465] INFO - query params : scan_consistency=request_plus&statement=SELECT+%2A+FROM+default+WHERE++join_yr+%3E+2010+and+join_yr+%3C+2014+ORDER+BY+_id+
[2019-12-10 23:22:38,475] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 210.118077ms
[2019-12-10 23:22:38,540] - [tuq_helper:214] INFO -  Analyzing Actual Result
[2019-12-10 23:22:38,540] - [tuq_helper:216] INFO -  Analyzing Expected Result
[2019-12-10 23:22:38,925] - [rest_client:3298] INFO - Flush for bucket 'default' was triggered
[2019-12-10 23:22:38,926] - [basetestcase:530] INFO - sleep for 5 secs.  ...
[2019-12-10 23:22:43,928] - [base_2i:328] INFO - Query : SELECT * FROM default WHERE  join_yr > 2010 and join_yr < 2014 ORDER BY _id 
[2019-12-10 23:22:43,928] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM default WHERE  join_yr > 2010 and join_yr < 2014 ORDER BY _id 
[2019-12-10 23:22:43,936] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+default+WHERE++join_yr+%3E+2010+and+join_yr+%3C+2014+ORDER+BY+_id+
[2019-12-10 23:22:43,943] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 4.317753ms
[2019-12-10 23:22:43,973] - [basetestcase:2436] INFO - list of index nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:22:43,977] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:22:43,977] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:22:45,113] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9000: True content: "/opt/build/ns_server/logs/n_0" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))).
[2019-12-10 23:22:45,114] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: zgrep "panic" "/opt/build/ns_server/logs/n_0"/indexer.log* | wc -l
[2019-12-10 23:22:45,294] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:22:45,312] - [basetestcase:2436] INFO - list of kv nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:22:45,316] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:22:45,316] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:22:46,466] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9000: True content: "/opt/build/ns_server/logs/n_0" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))).
[2019-12-10 23:22:46,467] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: zgrep "panic" "/opt/build/ns_server/logs/n_0"/projector.log* | wc -l
[2019-12-10 23:22:46,640] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:22:46,658] - [basetestcase:2436] INFO - list of n1ql nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:22:46,659] - [tuq_helper:516] INFO - CHECK FOR PRIMARY INDEXES
[2019-12-10 23:22:46,659] - [tuq_helper:523] INFO - DROP PRIMARY INDEX ON default USING GSI
[2019-12-10 23:22:46,659] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2019-12-10 23:22:46,667] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2019-12-10 23:22:46,679] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 9.214679ms
[2019-12-10 23:22:46,679] - [tuq_helper:147] INFO - RUN QUERY DROP PRIMARY INDEX ON default USING GSI
[2019-12-10 23:22:46,686] - [rest_client:3465] INFO - query params : statement=DROP+PRIMARY+INDEX+ON+default+USING+GSI
[2019-12-10 23:22:46,784] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 94.204353ms
[2019-12-10 23:22:46,784] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: rm -f /opt/couchbase/var/lib/couchbase/meta;rm -f /tmp/log_upr_client.sock
[2019-12-10 23:22:46,952] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:22:46,953] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: killall cbq-engine
[2019-12-10 23:22:47,115] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:22:47,116] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: killall tuqtng
[2019-12-10 23:22:47,324] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:22:47,325] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: killall indexer
[2019-12-10 23:22:47,525] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:22:47,552] - [basetestcase:397] INFO - ------- Cluster statistics -------
[2019-12-10 23:22:47,553] - [basetestcase:399] INFO - 127.0.0.1:9000 => {'swap_mem_used': 0, 'cpu_utilization': 19.14285714285714, 'mem_free': 13257060352, 'swap_mem_total': 0, 'mem_total': 15466930176, 'services': [u'index', u'kv', u'n1ql']}
[2019-12-10 23:22:47,553] - [basetestcase:400] INFO - --- End of cluster statistics ---
[2019-12-10 23:22:47,553] - [basetestcase:462] INFO - ==============  basetestcase cleanup was started for test #14 test_flush_bucket_and_query ==============
[2019-12-10 23:22:47,604] - [bucket_helper:143] INFO - deleting existing buckets [u'default'] on 127.0.0.1
[2019-12-10 23:22:47,604] - [bucket_helper:145] INFO - remove bucket default ...
[2019-12-10 23:22:48,876] - [bucket_helper:159] INFO - deleted bucket : default from 127.0.0.1
[2019-12-10 23:22:48,876] - [bucket_helper:237] INFO - waiting for bucket deletion to complete....
[2019-12-10 23:22:48,882] - [rest_client:133] INFO - node 127.0.0.1 existing buckets : []
[2019-12-10 23:22:48,882] - [bucket_helper:169] INFO - sleep 2 seconds to make sure all buckets ([u'default']) were deleted completely.
[2019-12-10 23:22:50,951] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9000
[2019-12-10 23:22:50,958] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9000 is running
[2019-12-10 23:22:50,965] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9001
[2019-12-10 23:22:50,971] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9001 is running
[2019-12-10 23:22:50,977] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9002
[2019-12-10 23:22:50,984] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9002 is running
[2019-12-10 23:22:50,990] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9003
[2019-12-10 23:22:50,996] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9003 is running
[2019-12-10 23:22:50,996] - [basetestcase:480] INFO - ==============  basetestcase cleanup was finished for test #14 test_flush_bucket_and_query ==============
ok

----------------------------------------------------------------------
Ran 1 test in 44.786s

OK
test_delete_create_bucket_and_query (2i.cluster_ops_2i.SecondaryIndexingClusterOpsTests) ... Cluster instance shutdown with force
summary so far suite 2i.indexscans_2i.SecondaryIndexingScanTests , pass 9 , fail 0
summary so far suite 2i.indexcreatedrop_2i.SecondaryIndexingCreateDropTests , pass 2 , fail 0
summary so far suite 2i.cluster_ops_2i.SecondaryIndexingClusterOpsTests , pass 3 , fail 0
testrunner logs, diags and results are available under /opt/build/testrunner/logs/testrunner-19-Dec-10_23-14-02/test_14
Logs will be stored at /opt/build/testrunner/logs/testrunner-19-Dec-10_23-14-02/test_15

./testrunner -i b/resources/dev-4-nodes-xdcr_n1ql_2i.ini -p makefile=True -t 2i.cluster_ops_2i.SecondaryIndexingClusterOpsTests.test_delete_create_bucket_and_query,groups=simple:and:no_orderby_groupby:range,dataset=default,doc-per-day=1,use_gsi_for_primary=True,use_gsi_for_secondary=True,GROUP=gsi

Test Input params:
{'doc-per-day': '1', 'use_gsi_for_primary': 'True', 'GROUP': 'gsi', 'use_gsi_for_secondary': 'True', 'gsi_type': 'forestdb', 'makefile': 'True', 'conf_file': 'conf/simple_2i_n1ql.conf', 'dataset': 'default', 'cluster_name': 'dev-4-nodes-xdcr_n1ql_2i', 'ini': 'b/resources/dev-4-nodes-xdcr_n1ql_2i.ini', 'groups': 'simple:and:no_orderby_groupby:range', 'case_number': 15, 'num_nodes': 4, 'logs_folder': '/opt/build/testrunner/logs/testrunner-19-Dec-10_23-14-02/test_15', 'spec': 'simple_2i_n1ql'}
[2019-12-10 23:22:51,057] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:22:51,057] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:22:52,137] - [rest_client:2127] INFO - Node version in cluster 6.5.0-4928-rel-enterprise
[2019-12-10 23:22:52,151] - [rest_client:2137] INFO - Node versions in cluster [u'6.5.0-4928-rel-enterprise']
[2019-12-10 23:22:52,151] - [basetestcase:218] INFO - ==============  basetestcase setup was started for test #15 test_delete_create_bucket_and_query==============
[2019-12-10 23:22:52,243] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9000
[2019-12-10 23:22:52,250] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9000 is running
[2019-12-10 23:22:52,256] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9001
[2019-12-10 23:22:52,263] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9001 is running
[2019-12-10 23:22:52,269] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9002
[2019-12-10 23:22:52,275] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9002 is running
[2019-12-10 23:22:52,281] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9003
[2019-12-10 23:22:52,287] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9003 is running
[2019-12-10 23:22:52,288] - [basetestcase:265] INFO - initializing cluster
[2019-12-10 23:22:53,086] - [task:132] INFO - server: ip:127.0.0.1 port:9000 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_0@cb.local', 'uptime': u'524', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9000', 'storage': [], 'moxi': 11211, 'port': u'9000', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12000, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'index', u'kv', u'n1ql'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13327507456, 'memoryTotal': 15466930176, 'memoryQuota': 7650, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:22:53,094] - [task:171] INFO - quota for index service will be 256 MB
[2019-12-10 23:22:53,094] - [task:173] INFO - set index quota to node 127.0.0.1 
[2019-12-10 23:22:53,094] - [rest_client:965] INFO - pools/default params : indexMemoryQuota=256
[2019-12-10 23:22:53,103] - [rest_client:953] INFO - pools/default params : memoryQuota=7650
[2019-12-10 23:22:53,123] - [rest_client:931] INFO - /node/controller/setupServices params on 127.0.0.1:9000:services=kv%2Cindex%2Cn1ql&password=asdasd&user=Administrator&hostname=127.0.0.1%3A9000
[2019-12-10 23:22:53,129] - [rest_client:840] ERROR - POST http://127.0.0.1:9000//node/controller/setupServices body: services=kv%2Cindex%2Cn1ql&password=asdasd&user=Administrator&hostname=127.0.0.1%3A9000 headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown ["cannot change node services after cluster is provisioned"] auth: Administrator:asdasd
[2019-12-10 23:22:53,130] - [rest_client:936] INFO - This node is already provisioned with services, we do not consider this as failure for test case
[2019-12-10 23:22:53,130] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9000:username=Administrator&password=asdasd&port=9000
[2019-12-10 23:22:53,189] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:22:53,190] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:22:54,226] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9000/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:22:54,389] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:22:54,397] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9000: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:22:54,398] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:22:54,404] - [rest_client:840] ERROR - POST http://127.0.0.1:9000/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:22:54,431] - [task:132] INFO - server: ip:127.0.0.1 port:9001 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_1@cb.local', 'uptime': u'524', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9001', 'storage': [], 'moxi': 11211, 'port': u'9001', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12002, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13327511552, 'memoryTotal': 15466930176, 'memoryQuota': 7906, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:22:54,441] - [rest_client:953] INFO - pools/default params : memoryQuota=7906
[2019-12-10 23:22:54,449] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9001:username=Administrator&password=asdasd&port=9001
[2019-12-10 23:22:54,511] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:22:54,511] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:22:55,554] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9001/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:22:55,714] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:22:55,723] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9001: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:22:55,723] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:22:55,733] - [rest_client:840] ERROR - POST http://127.0.0.1:9001/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:22:55,759] - [task:132] INFO - server: ip:127.0.0.1 port:9002 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_2@cb.local', 'uptime': u'529', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9002', 'storage': [], 'moxi': 11211, 'port': u'9002', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12004, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13371899904, 'memoryTotal': 15466930176, 'memoryQuota': 7906, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:22:55,769] - [rest_client:953] INFO - pools/default params : memoryQuota=7906
[2019-12-10 23:22:55,778] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9002:username=Administrator&password=asdasd&port=9002
[2019-12-10 23:22:55,836] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:22:55,837] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:22:56,884] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9002/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:22:57,047] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:22:57,055] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9002: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:22:57,056] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:22:57,061] - [rest_client:840] ERROR - POST http://127.0.0.1:9002/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:22:57,087] - [task:132] INFO - server: ip:127.0.0.1 port:9003 ssh_username:Administrator, nodes/self: {'ip': u'127.0.0.1', 'availableStorage': [], 'rest_username': '', 'id': u'n_3@cb.local', 'uptime': u'529', 'mcdMemoryReserved': 11800, 'storageTotalRam': 14750, 'hostname': u'127.0.0.1:9003', 'storage': [], 'moxi': 11211, 'port': u'9003', 'version': u'6.5.0-4928-rel-enterprise', 'memcached': 12006, 'status': u'healthy', 'clusterCompatibility': 393221, 'curr_items': 0, 'services': [u'kv'], 'rest_password': '', 'clusterMembership': u'active', 'memoryFree': 13373599744, 'memoryTotal': 15466930176, 'memoryQuota': 7906, 'mcdMemoryAllocated': 11800, 'os': u'x86_64-unknown-linux-gnu', 'ports': []}
[2019-12-10 23:22:57,097] - [rest_client:953] INFO - pools/default params : memoryQuota=7906
[2019-12-10 23:22:57,105] - [rest_client:864] INFO - settings/web params on 127.0.0.1:9003:username=Administrator&password=asdasd&port=9003
[2019-12-10 23:22:57,165] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:22:57,165] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:22:58,210] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9003/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:22:58,369] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:22:58,377] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9003: True content: [6,5] command: cluster_compat_mode:get_compat_version().
[2019-12-10 23:22:58,377] - [rest_client:988] INFO - settings/indexes params : storageMode=forestdb
[2019-12-10 23:22:58,383] - [rest_client:840] ERROR - POST http://127.0.0.1:9003/settings/indexes body: storageMode=forestdb headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': '*/*', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=\n'} error: 400 reason: unknown {"errors":{"storageMode":"The value must be one of the following: [plasma,memory_optimized]"}} auth: Administrator:asdasd
[2019-12-10 23:22:58,413] - [basetestcase:2218] INFO - **** add built-in 'cbadminbucket' user to node 127.0.0.1 ****
[2019-12-10 23:22:58,520] - [basetestcase:530] INFO - sleep for 5 secs.  ...
[2019-12-10 23:23:03,526] - [basetestcase:2223] INFO - **** add 'admin' role to 'cbadminbucket' user ****
[2019-12-10 23:23:03,552] - [basetestcase:291] INFO - done initializing cluster
[2019-12-10 23:23:03,557] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:23:03,557] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:23:04,607] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: curl http://Administrator:asdasd@localhost:9000/diag/eval -X POST -d 'ns_config:set(allow_nonlocal_eval, true).'
[2019-12-10 23:23:04,774] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:23:04,775] - [basetestcase:2770] INFO - Enabled diag/eval for non-local hosts from 127.0.0.1
[2019-12-10 23:23:05,443] - [rest_client:2430] INFO - http://127.0.0.1:9000/pools/default/buckets with param: bucketType=membase&evictionPolicy=valueOnly&threadsNumber=3&ramQuotaMB=7650&authType=sasl&name=default&flushEnabled=1&replicaNumber=1&compressionMode=passive&saslPassword=None&replicaIndex=1
[2019-12-10 23:23:05,454] - [rest_client:2455] INFO - 0.01 seconds to create bucket default
[2019-12-10 23:23:05,460] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:23:05,611] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:23:05,692] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:23:05,745] - [task:351] WARNING - vbucket map not ready after try 0
[2019-12-10 23:23:05,745] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:23:05,834] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:23:05,916] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:23:06,186] - [task:351] WARNING - vbucket map not ready after try 1
[2019-12-10 23:23:06,186] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:23:06,266] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:23:06,348] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:23:06,389] - [task:351] WARNING - vbucket map not ready after try 2
[2019-12-10 23:23:06,389] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:23:06,466] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:23:06,554] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:23:06,614] - [task:351] WARNING - vbucket map not ready after try 3
[2019-12-10 23:23:06,614] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:23:06,825] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:23:06,909] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:23:06,953] - [task:351] WARNING - vbucket map not ready after try 4
[2019-12-10 23:23:06,953] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:23:07,031] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:23:07,110] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:23:07,153] - [task:351] WARNING - vbucket map not ready after try 5
[2019-12-10 23:23:07,200] - [basetestcase:383] INFO - ==============  basetestcase setup was finished for test #15 test_delete_create_bucket_and_query ==============
[2019-12-10 23:23:07,223] - [basetestcase:397] INFO - ------- Cluster statistics -------
[2019-12-10 23:23:07,224] - [basetestcase:399] INFO - 127.0.0.1:9000 => {'swap_mem_used': 0, 'cpu_utilization': 15.72164948453608, 'mem_free': 13374730240, 'swap_mem_total': 0, 'mem_total': 15466930176, 'services': [u'index', u'kv', u'n1ql']}
[2019-12-10 23:23:07,224] - [basetestcase:400] INFO - --- End of cluster statistics ---
[2019-12-10 23:23:07,236] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is warmup
[2019-12-10 23:23:07,236] - [newtuq:22] INFO - Waiting for cluster to become healthy
[2019-12-10 23:23:07,237] - [basetestcase:530] INFO - sleep for 5 secs.  ...
[2019-12-10 23:23:12,248] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:23:12,262] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is healthy
[2019-12-10 23:23:12,262] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:23:12,275] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is healthy
[2019-12-10 23:23:12,275] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:23:12,290] - [newtuq:20] INFO - Initial status of 127.0.0.1 cluster is healthy
[2019-12-10 23:23:12,290] - [newtuq:25] INFO - current status of 127.0.0.1  is healthy
[2019-12-10 23:23:12,302] - [basetestcase:2436] INFO - list of index nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:23:12,322] - [rest_client:1706] INFO - {'indexer.settings.storage_mode': 'forestdb'} set
[2019-12-10 23:23:12,359] - [rest_client:1706] INFO - {'indexer.api.enableTestServer': True} set
[2019-12-10 23:23:12,362] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:23:12,363] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:23:14,873] - [basetestcase:2545] INFO - create 2016 to default documents...
[2019-12-10 23:23:14,928] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:23:16,488] - [basetestcase:2557] INFO - LOAD IS FINISHED
[2019-12-10 23:23:16,527] - [basetestcase:2436] INFO - list of n1ql nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:23:16,527] - [newtuq:83] INFO - ip:127.0.0.1 port:9000 ssh_username:Administrator
[2019-12-10 23:23:16,527] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2019-12-10 23:23:16,535] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2019-12-10 23:23:16,626] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 86.629559ms
[2019-12-10 23:23:16,626] - [tuq_helper:147] INFO - RUN QUERY CREATE PRIMARY INDEX ON default  USING GSI
[2019-12-10 23:23:16,636] - [rest_client:3465] INFO - query params : statement=CREATE+PRIMARY+INDEX+ON+default++USING+GSI
[2019-12-10 23:23:17,418] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 779.375425ms
[2019-12-10 23:23:17,419] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2019-12-10 23:23:17,432] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2019-12-10 23:23:17,446] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 11.134628ms
[2019-12-10 23:23:17,485] - [basetestcase:2436] INFO - list of n1ql nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:23:18,472] - [tuq_helper:147] INFO - RUN QUERY CREATE INDEX employee21b302f160cc4b01b790d3a0fc482d3djoin_yr ON default(join_yr) USING GSI  WITH {'defer_build': True}
[2019-12-10 23:23:18,479] - [rest_client:3465] INFO - query params : statement=CREATE+INDEX+employee21b302f160cc4b01b790d3a0fc482d3djoin_yr+ON+default%28join_yr%29+USING+GSI++WITH+%7B%27defer_build%27%3A+True%7D
[2019-12-10 23:23:18,765] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 280.993328ms
[2019-12-10 23:23:18,765] - [base_2i:109] INFO - BUILD INDEX on default(employee21b302f160cc4b01b790d3a0fc482d3djoin_yr) USING GSI
[2019-12-10 23:23:19,766] - [tuq_helper:147] INFO - RUN QUERY BUILD INDEX on default(employee21b302f160cc4b01b790d3a0fc482d3djoin_yr) USING GSI
[2019-12-10 23:23:19,774] - [rest_client:3465] INFO - query params : statement=BUILD+INDEX+on+default%28employee21b302f160cc4b01b790d3a0fc482d3djoin_yr%29+USING+GSI
[2019-12-10 23:23:19,856] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 79.390342ms
[2019-12-10 23:23:19,857] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee21b302f160cc4b01b790d3a0fc482d3djoin_yr'
[2019-12-10 23:23:19,867] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee21b302f160cc4b01b790d3a0fc482d3djoin_yr%27
[2019-12-10 23:23:19,882] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 11.126851ms
[2019-12-10 23:23:20,884] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee21b302f160cc4b01b790d3a0fc482d3djoin_yr'
[2019-12-10 23:23:20,891] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee21b302f160cc4b01b790d3a0fc482d3djoin_yr%27
[2019-12-10 23:23:20,906] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 13.004894ms
[2019-12-10 23:23:21,908] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee21b302f160cc4b01b790d3a0fc482d3djoin_yr'
[2019-12-10 23:23:21,915] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee21b302f160cc4b01b790d3a0fc482d3djoin_yr%27
[2019-12-10 23:23:21,933] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 14.059241ms
[2019-12-10 23:23:22,934] - [tuq_helper:147] INFO - RUN QUERY EXPLAIN SELECT * FROM default WHERE  join_yr > 2010 and join_yr < 2014 ORDER BY _id 
[2019-12-10 23:23:22,942] - [rest_client:3465] INFO - query params : statement=EXPLAIN+SELECT+%2A+FROM+default+WHERE++join_yr+%3E+2010+and+join_yr+%3C+2014+ORDER+BY+_id+
[2019-12-10 23:23:22,949] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 4.915106ms
[2019-12-10 23:23:22,950] - [base_2i:261] INFO - {u'status': u'success', u'metrics': {u'elapsedTime': u'4.915106ms', u'executionTime': u'4.794497ms', u'resultSize': 895, u'resultCount': 1}, u'results': [{u'text': u'SELECT * FROM default WHERE  join_yr > 2010 and join_yr < 2014 ORDER BY _id', u'plan': {u'#operator': u'Sequence', u'~children': [{u'#operator': u'Sequence', u'~children': [{u'index': u'employee21b302f160cc4b01b790d3a0fc482d3djoin_yr', u'namespace': u'default', u'keyspace': u'default', u'#operator': u'IndexScan3', u'index_projection': {u'primary_key': True}, u'spans': [{u'range': [{u'high': u'2014', u'low': u'2010', u'inclusion': 0}], u'exact': True}], u'using': u'gsi', u'index_id': u'18d97664519917b9'}, {u'keyspace': u'default', u'#operator': u'Fetch', u'namespace': u'default'}, {u'#operator': u'Parallel', u'~child': {u'#operator': u'Sequence', u'~children': [{u'#operator': u'Filter', u'condition': u'((2010 < (`default`.`join_yr`)) and ((`default`.`join_yr`) < 2014))'}, {u'#operator': u'InitialProject', u'result_terms': [{u'expr': u'self', u'star': True}]}]}}]}, {u'#operator': u'Order', u'sort_terms': [{u'expr': u'(`default`.`_id`)'}]}, {u'#operator': u'FinalProject'}]}}], u'requestID': u'b6754927-1f59-41c3-a480-52f36803e0ff', u'signature': u'json'}
[2019-12-10 23:23:22,951] - [tuq_generators:70] INFO - FROM clause ===== is default
[2019-12-10 23:23:22,951] - [tuq_generators:72] INFO - WHERE clause ===== is   doc["join_yr"] > 2010 and doc["join_yr"] < 2014 
[2019-12-10 23:23:22,951] - [tuq_generators:74] INFO - UNNEST clause ===== is None
[2019-12-10 23:23:22,952] - [tuq_generators:76] INFO - SELECT clause ===== is {"*" : doc,}
[2019-12-10 23:23:22,952] - [tuq_generators:417] INFO - ORDER clause ========= is doc["_id"],
[2019-12-10 23:23:22,952] - [tuq_generators:417] INFO - ORDER clause ========= is doc["_id"],
[2019-12-10 23:23:23,054] - [tuq_generators:417] INFO - ORDER clause ========= is doc["_id"],
[2019-12-10 23:23:23,068] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM default WHERE  join_yr > 2010 and join_yr < 2014 ORDER BY _id 
[2019-12-10 23:23:23,076] - [rest_client:3465] INFO - query params : scan_consistency=request_plus&statement=SELECT+%2A+FROM+default+WHERE++join_yr+%3E+2010+and+join_yr+%3C+2014+ORDER+BY+_id+
[2019-12-10 23:23:23,336] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 229.632818ms
[2019-12-10 23:23:23,397] - [tuq_helper:214] INFO -  Analyzing Actual Result
[2019-12-10 23:23:23,398] - [tuq_helper:216] INFO -  Analyzing Expected Result
[2019-12-10 23:23:23,791] - [basetestcase:530] INFO - sleep for 2 secs.  ...
[2019-12-10 23:23:26,304] - [rest_client:2430] INFO - http://127.0.0.1:9000/pools/default/buckets with param: bucketType=membase&evictionPolicy=valueOnly&threadsNumber=3&ramQuotaMB=7650&authType=sasl&name=default&flushEnabled=1&replicaNumber=1&compressionMode=passive&saslPassword=None&replicaIndex=1
[2019-12-10 23:23:26,326] - [rest_client:2455] INFO - 0.02 seconds to create bucket default
[2019-12-10 23:23:26,335] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:23:26,463] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:23:26,547] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:23:26,591] - [task:351] WARNING - vbucket map not ready after try 0
[2019-12-10 23:23:26,592] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:23:26,917] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:23:26,997] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:23:27,039] - [task:351] WARNING - vbucket map not ready after try 1
[2019-12-10 23:23:27,039] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:23:27,122] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:23:27,217] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:23:27,261] - [task:351] WARNING - vbucket map not ready after try 2
[2019-12-10 23:23:27,261] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:23:27,341] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:23:27,596] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:23:27,637] - [task:351] WARNING - vbucket map not ready after try 3
[2019-12-10 23:23:27,637] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:23:27,714] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:23:27,793] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:23:27,835] - [task:351] WARNING - vbucket map not ready after try 4
[2019-12-10 23:23:27,835] - [bucket_helper:344] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2019-12-10 23:23:27,915] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:23:27,994] - [data_helper:304] INFO - creating direct client 127.0.0.1:12000 default
[2019-12-10 23:23:28,035] - [task:351] WARNING - vbucket map not ready after try 5
[2019-12-10 23:23:28,051] - [basetestcase:530] INFO - sleep for 2 secs.  ...
[2019-12-10 23:23:30,061] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9000
[2019-12-10 23:23:30,067] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9000 is running
[2019-12-10 23:23:30,074] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9001
[2019-12-10 23:23:30,080] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9001 is running
[2019-12-10 23:23:30,086] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9002
[2019-12-10 23:23:30,093] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9002 is running
[2019-12-10 23:23:30,099] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9003
[2019-12-10 23:23:30,106] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9003 is running
[2019-12-10 23:23:30,122] - [basetestcase:2436] INFO - list of n1ql nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:23:30,122] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee21b302f160cc4b01b790d3a0fc482d3djoin_yr'
[2019-12-10 23:23:30,128] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee21b302f160cc4b01b790d3a0fc482d3djoin_yr%27
[2019-12-10 23:23:30,153] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 21.40699ms
[2019-12-10 23:23:30,154] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee21b302f160cc4b01b790d3a0fc482d3djoin_yr'
[2019-12-10 23:23:30,161] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee21b302f160cc4b01b790d3a0fc482d3djoin_yr%27
[2019-12-10 23:23:30,171] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 7.051893ms
[2019-12-10 23:23:30,183] - [basetestcase:2436] INFO - list of index nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:23:30,216] - [basetestcase:2436] INFO - list of index nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:23:30,219] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:23:30,219] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:23:31,307] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9000: True content: "/opt/build/ns_server/logs/n_0" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))).
[2019-12-10 23:23:31,307] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: zgrep "panic" "/opt/build/ns_server/logs/n_0"/indexer.log* | wc -l
[2019-12-10 23:23:31,465] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:23:31,482] - [basetestcase:2436] INFO - list of kv nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:23:31,485] - [remote_util:232] INFO - connecting to 127.0.0.1 with username:Administrator 
[2019-12-10 23:23:31,486] - [remote_util:277] INFO - Connected to 127.0.0.1
[2019-12-10 23:23:32,542] - [rest_client:1521] INFO - /diag/eval status on 127.0.0.1:9000: True content: "/opt/build/ns_server/logs/n_0" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))).
[2019-12-10 23:23:32,543] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: zgrep "panic" "/opt/build/ns_server/logs/n_0"/projector.log* | wc -l
[2019-12-10 23:23:32,700] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:23:32,719] - [basetestcase:2436] INFO - list of n1ql nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2019-12-10 23:23:32,719] - [tuq_helper:516] INFO - CHECK FOR PRIMARY INDEXES
[2019-12-10 23:23:32,720] - [tuq_helper:523] INFO - DROP PRIMARY INDEX ON default USING GSI
[2019-12-10 23:23:32,720] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2019-12-10 23:23:32,727] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2019-12-10 23:23:32,740] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 10.844187ms
[2019-12-10 23:23:32,740] - [tuq_helper:523] INFO - DROP PRIMARY INDEX ON default USING GSI
[2019-12-10 23:23:32,741] - [tuq_helper:147] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2019-12-10 23:23:32,747] - [rest_client:3465] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2019-12-10 23:23:32,757] - [tuq_helper:171] INFO - TOTAL ELAPSED TIME: 7.000917ms
[2019-12-10 23:23:32,757] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: rm -f /opt/couchbase/var/lib/couchbase/meta;rm -f /tmp/log_upr_client.sock
[2019-12-10 23:23:32,916] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:23:32,917] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: killall cbq-engine
[2019-12-10 23:23:33,071] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:23:33,072] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: killall tuqtng
[2019-12-10 23:23:33,275] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:23:33,276] - [remote_util:3156] INFO - running command.raw on 127.0.0.1: killall indexer
[2019-12-10 23:23:33,454] - [remote_util:3193] INFO - command executed successfully
[2019-12-10 23:23:33,473] - [basetestcase:397] INFO - ------- Cluster statistics -------
[2019-12-10 23:23:33,474] - [basetestcase:399] INFO - 127.0.0.1:9000 => {'swap_mem_used': 0, 'cpu_utilization': 1.017811704834606, 'mem_free': 13299318784, 'swap_mem_total': 0, 'mem_total': 15466930176, 'services': [u'index', u'kv', u'n1ql']}
[2019-12-10 23:23:33,474] - [basetestcase:400] INFO - --- End of cluster statistics ---
[2019-12-10 23:23:33,474] - [basetestcase:462] INFO - ==============  basetestcase cleanup was started for test #15 test_delete_create_bucket_and_query ==============
[2019-12-10 23:23:33,531] - [bucket_helper:143] INFO - deleting existing buckets [u'default'] on 127.0.0.1
[2019-12-10 23:23:33,532] - [bucket_helper:145] INFO - remove bucket default ...
[2019-12-10 23:23:33,906] - [bucket_helper:159] INFO - deleted bucket : default from 127.0.0.1
[2019-12-10 23:23:33,906] - [bucket_helper:237] INFO - waiting for bucket deletion to complete....
[2019-12-10 23:23:33,913] - [rest_client:133] INFO - node 127.0.0.1 existing buckets : []
[2019-12-10 23:23:33,913] - [bucket_helper:169] INFO - sleep 2 seconds to make sure all buckets ([u'default']) were deleted completely.
[2019-12-10 23:23:35,980] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9000
[2019-12-10 23:23:35,986] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9000 is running
[2019-12-10 23:23:35,992] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9001
[2019-12-10 23:23:35,998] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9001 is running
[2019-12-10 23:23:36,005] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9002
[2019-12-10 23:23:36,011] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9002 is running
[2019-12-10 23:23:36,018] - [cluster_helper:82] INFO - waiting for ns_server @ 127.0.0.1:9003
[2019-12-10 23:23:36,025] - [cluster_helper:86] INFO - ns_server @ 127.0.0.1:9003 is running
[2019-12-10 23:23:36,025] - [basetestcase:480] INFO - ==============  basetestcase cleanup was finished for test #15 test_delete_create_bucket_and_query ==============
ok

----------------------------------------------------------------------
Ran 1 test in 44.984s

OK
Cluster instance shutdown with force
summary so far suite 2i.indexscans_2i.SecondaryIndexingScanTests , pass 9 , fail 0
summary so far suite 2i.indexcreatedrop_2i.SecondaryIndexingCreateDropTests , pass 2 , fail 0
summary so far suite 2i.cluster_ops_2i.SecondaryIndexingClusterOpsTests , pass 4 , fail 0
testrunner logs, diags and results are available under /opt/build/testrunner/logs/testrunner-19-Dec-10_23-14-02/test_15
Run after suite setup for 2i.cluster_ops_2i.SecondaryIndexingClusterOpsTests.test_delete_create_bucket_and_query
2i.indexscans_2i.SecondaryIndexingScanTests.test_multi_create_query_explain_drop_index  pass
2i.indexscans_2i.SecondaryIndexingScanTests.test_multi_create_query_explain_drop_index  pass
2i.indexscans_2i.SecondaryIndexingScanTests.test_multi_create_query_explain_drop_index  pass
2i.indexscans_2i.SecondaryIndexingScanTests.test_multi_create_query_explain_drop_index  pass
2i.indexscans_2i.SecondaryIndexingScanTests.test_multi_create_query_explain_drop_index  pass
2i.indexscans_2i.SecondaryIndexingScanTests.test_multi_create_query_explain_drop_index  pass
2i.indexscans_2i.SecondaryIndexingScanTests.test_multi_create_query_explain_drop_index  pass
2i.indexscans_2i.SecondaryIndexingScanTests.test_multi_create_query_explain_drop_index  pass
2i.indexscans_2i.SecondaryIndexingScanTests.test_multi_create_query_explain_drop_index  pass
2i.indexcreatedrop_2i.SecondaryIndexingCreateDropTests.test_multi_create_drop_index  pass
2i.indexcreatedrop_2i.SecondaryIndexingCreateDropTests.test_multi_create_drop_index  pass
2i.cluster_ops_2i.SecondaryIndexingClusterOpsTests.test_remove_bucket_and_query  pass
2i.cluster_ops_2i.SecondaryIndexingClusterOpsTests.test_change_bucket_properties  pass
2i.cluster_ops_2i.SecondaryIndexingClusterOpsTests.test_flush_bucket_and_query  pass
2i.cluster_ops_2i.SecondaryIndexingClusterOpsTests.test_delete_create_bucket_and_query  pass
Thread Cluster_Thread was not properly terminated, will be terminated now.
Thread Cluster_Thread was not properly terminated, will be terminated now.
Thread Cluster_Thread was not properly terminated, will be terminated now.
Thread Cluster_Thread was not properly terminated, will be terminated now.
Thread Cluster_Thread was not properly terminated, will be terminated now.
Thread Cluster_Thread was not properly terminated, will be terminated now.
Thread Cluster_Thread was not properly terminated, will be terminated now.
Thread Cluster_Thread was not properly terminated, will be terminated now.
Thread Cluster_Thread was not properly terminated, will be terminated now.
Thread Cluster_Thread was not properly terminated, will be terminated now.
Thread Cluster_Thread was not properly terminated, will be terminated now.
Thread Cluster_Thread was not properly terminated, will be terminated now.
Thread Cluster_Thread was not properly terminated, will be terminated now.
Thread Cluster_Thread was not properly terminated, will be terminated now.
Thread Cluster_Thread was not properly terminated, will be terminated now.
Thread Cluster_Thread was not properly terminated, will be terminated now.
scripts/start_cluster_and_run_tests.sh: line 85: 27271 Terminated              COUCHBASE_NUM_VBUCKETS=64 python ./cluster_run --nodes=$servers_count &> $wd/cluster_run.log  (wd: /opt/build/ns_server)

Testing Failed: Required test failed

FAIL	github.com/couchbase/indexing/secondary/tests/functionaltests	7284.775s
--- FAIL: TestIndexNodeRebalanceOut (135.25s)
--- FAIL: TestFailoverAndRebalance (0.02s)
Version: versions-10.12.2019-20.06.cfg
Build Log: make-10.12.2019-20.06.log
Server Log: logs-10.12.2019-20.06.tar.gz

Finished