Building

Updating tree for run 06.04.2016-07.42
Switching indexing to unstable
indexing is at 6b702c3, changes since last good build: none
query is at 6320d4c, changes since last good build: none
couchdb is at 58b8edf, changes since last good build: none
couchstore is at 9e90a38, changes since last good build: none
forestdb is at 93b9365, changes since last good build: 
 93b9365 Filemgr_openlock to be acquired for a hash table scan
ep-engine is at 75487f0, changes since last good build: none
testrunner is at d8ecb26, changes since last good build: none
Building main product
Testing 32-bit compilation

Testing

Using storage type: fdb

Simple Test

Apr 06 07:54:53 test_MB_12751 (regressions.regressiontest.RegressionTests) ... ok
Apr 06 07:55:18 test_MB_14288 (regressions.regressiontest.RegressionTests) ... ok
Apr 06 07:57:51 rebalance_in_with_ops (rebalance.rebalancein.RebalanceInTests) ... ok
Apr 06 07:58:33 do_warmup_100k (memcapable.WarmUpMemcachedTest) ... ok
Apr 06 08:00:02 test_view_ops (view.createdeleteview.CreateDeleteViewTests) ... ok
Apr 06 08:00:02 summary so far suite regressions.regressiontest.RegressionTests , pass 1 , fail 0
Apr 06 08:00:02 summary so far suite regressions.regressiontest.RegressionTests , pass 2 , fail 0
Apr 06 08:00:02 summary so far suite regressions.regressiontest.RegressionTests , pass 2 , fail 0
Apr 06 08:00:02 summary so far suite rebalance.rebalancein.RebalanceInTests , pass 1 , fail 0
Apr 06 08:00:02 summary so far suite regressions.regressiontest.RegressionTests , pass 2 , fail 0
Apr 06 08:00:02 summary so far suite rebalance.rebalancein.RebalanceInTests , pass 1 , fail 0
Apr 06 08:00:02 summary so far suite memcapable.WarmUpMemcachedTest , pass 1 , fail 0
Apr 06 08:05:37 Logs will be stored at /opt/build/testrunner/ltest_employee_dataset_startkey_endkey_queries_rebalance_in (view.viewquerytests.ViewQueryTests) ... ok
Apr 06 08:06:20 test_simple_dataset_stale_queries_data_modification (view.viewquerytests.ViewQueryTests) ... ok
Apr 06 08:08:32 load_with_ops (xdcr.uniXDCR.unidirectional) ... ok
Apr 06 08:08:32 load_with_failover (xdcr.uniXDCR.unidirectional) ... ogs/testrunner-16-Apr-06_07-54-19/test_5
Apr 06 08:08:32 summary so far suite regressions.regressiontest.RegressionTests , pass 2 , fail 0
Apr 06 08:08:32 summary so far suite rebalance.rebalancein.RebalanceInTests , pass 1 , fail 0
Apr 06 08:08:32 summary so far suite memcapable.WarmUpMemcachedTest , pass 1 , fail 0
Apr 06 08:08:32 summary so far suite view.createdeleteview.CreateDeleteViewTests , pass 1 , fail 0
Apr 06 08:08:32 summary so far suite regressions.regressiontest.RegressionTests , pass 2 , fail 0
Apr 06 08:08:32 summary so far suite rebalance.rebalancein.RebalanceInTests , pass 1 , fail 0
Apr 06 08:08:32 summary so far suite memcapable.WarmUpMemcachedTest , pass 1 , fail 0
Apr 06 08:08:32 summary so far suite view.createdeleteview.CreateDeleteViewTests , pass 1 , fail 0
Apr 06 08:08:32 summary so far suite view.viewquerytests.ViewQueryTests , pass 1 , fail 0
Apr 06 08:08:32 summary so far suite regressions.regressiontest.RegressionTests , pass 2 , fail 0
Apr 06 08:08:32 summary so far suite rebalance.rebalancein.RebalanceInTests , pass 1 , fail 0
Apr 06 08:08:32 summary so far suite memcapable.WarmUpMemcachedTest , pass 1 , fail 0
Apr 06 08:08:32 summary so far suite view.createdeleteview.CreateDeleteViewTests , pass 1 , fail 0
Apr 06 08:08:32 summary so far suite view.viewquerytests.ViewQueryTests , pass 2 , fail 0
Apr 06 08:11:00 {'doc-ops': 'update-delete', 'log_level': 'CRITICAL'ok
Apr 06 08:11:00 summary so far suite regressions.regressiontest.RegressionTests , pass 2 , fail 0
Apr 06 08:11:00 summary so far suite rebalance.rebalancein.RebalanceInTests , pass 1 , fail 0
Apr 06 08:11:00 summary so far suite memcapable.WarmUpMemcachedTest , pass 1 , fail 0
Apr 06 08:11:00 summary so far suite view.createdeleteview.CreateDeleteViewTests , pass 1 , fail 0
Apr 06 08:11:00 summary so far suite view.viewquerytests.ViewQueryTests , pass 2 , fail 0
Apr 06 08:11:00 summary so far suite xdcr.uniXDCR.unidirectional , pass 1 , fail 0
Apr 06 08:11:00 ./testrunner -i b/resources/dev-4-nodes-xdcr.ini makefile=True,log_level=CRITICAL -t xdcr.uniXDCR.unidirectional.load_with_failover,replicas=1,items=10000,ctopology=chain,rdirection=unidirection,doc-ops=update-delete,failover=source
Apr 06 08:11:00 {'doc-ops': 'update-delete', 'log_level': 'CRITICAL', 'replicas': '1', 'items': '10000', 'failover': 'source', 'makefile': 'True', 'conf_file': 'conf/simple.conf', 'num_nodes': 4, 'cluster_name': 'dev-4-nodes-xdcr', 'ctopology': 'chain', 'rdirection': 'unidirection', 'ini': 'b/resources/dev-4-nodes-xdcr.ini', 'case_number': 9, 'logs_folder': '/opt/build/testrunner/logs/testrunner-16-Apr-06_07-54-19/test_9', 'spec': 'simple'}
Apr 06 08:11:00 summary so far suite regressions.regressiontest.RegressionTests , pass 2 , fail 0
Apr 06 08:11:00 summary so far suite rebalance.rebalancein.RebalanceInTests , pass 1 , fail 0
Apr 06 08:11:00 summary so far suite memcapable.WarmUpMemcachedTest , pass 1 , fail 0
Apr 06 08:11:00 summary so far suite view.createdeleteview.CreateDeleteViewTests , pass 1 , fail 0
Apr 06 08:11:00 summary so far suite view.viewquerytests.ViewQueryTests , pass 2 , fail 0
Apr 06 08:11:00 summary so far suite xdcr.uniXDCR.unidirectional , pass 2 , fail 0
Apr 06 08:11:00 Run after suite setup for xdcr.uniXDCR.unidirectional.load_with_failover
Apr 06 08:11:00 regressions.regressiontest.RegressionTests.test_MB_12751  pass
Apr 06 08:11:00 regressions.regressiontest.RegressionTests.test_MB_14288  pass
Apr 06 08:11:00 rebalance.rebalancein.RebalanceInTests.rebalance_in_with_ops  pass
Apr 06 08:11:00 memcapable.WarmUpMemcachedTest.do_warmup_100k  pass
Apr 06 08:11:00 view.createdeleteview.CreateDeleteViewTests.test_view_ops  pass
Apr 06 08:11:00 view.viewquerytests.ViewQueryTests.test_employee_dataset_startkey_endkey_queries_rebalance_in  pass
Apr 06 08:11:00 view.viewquerytests.ViewQueryTests.test_simple_dataset_stale_queries_data_modification  pass
Apr 06 08:11:00 xdcr.uniXDCR.unidirectional.load_with_ops  pass
Apr 06 08:11:00 xdcr.uniXDCR.unidirectional.load_with_failover  pass

Functional tests

Starting server: attempt 1
Starting functional tests

Unit tests

=== RUN   TestMerger
--- PASS: TestMerger (0.02s)
=== RUN   TestInsert
--- PASS: TestInsert (0.00s)
=== RUN   TestInsertPerf
16000000 items took 29.849681152s -> 536019.1259171277 items/s conflicts 20
--- PASS: TestInsertPerf (29.85s)
=== RUN   TestGetPerf
16000000 items took 9.665064666s -> 1.6554467613946951e+06 items/s
--- PASS: TestGetPerf (10.90s)
=== RUN   TestGetRangeSplitItems
node_count             = 1000000
soft_deletes           = 0
read_conflicts         = 0
insert_conflicts       = 0
next_pointers_per_node = 1.3351
memory_used            = 37360992
node_allocs            = 1000000
node_frees             = 0

level_node_distribution:
level0 => 749144
level1 => 187776
level2 => 47222
level3 => 11903
level4 => 2974
level5 => 735
level6 => 182
level7 => 48
level8 => 12
level9 => 2
level10 => 2
level11 => 0
level12 => 0
level13 => 0
level14 => 0
level15 => 0
level16 => 0
level17 => 0
level18 => 0
level19 => 0
level20 => 0
level21 => 0
level22 => 0
level23 => 0
level24 => 0
level25 => 0
level26 => 0
level27 => 0
level28 => 0
level29 => 0
level30 => 0
level31 => 0
level32 => 0

Split range keys [35911 108875 357987 462344 598654 671258 781160]
No of items in each range [35911 72964 249112 104357 136310 72604 109902 218840]
--- PASS: TestGetRangeSplitItems (1.24s)
=== RUN   TestBuilder
node_count             = 50000000
soft_deletes           = 0
read_conflicts         = 0
insert_conflicts       = 0
next_pointers_per_node = 1.3333
memory_used            = 1866650928
node_allocs            = 50000000
node_frees             = 0

level_node_distribution:
level0 => 37500395
level1 => 9374070
level2 => 2344403
level3 => 586130
level4 => 146617
level5 => 36338
level6 => 9033
level7 => 2299
level8 => 539
level9 => 123
level10 => 39
level11 => 9
level12 => 5
level13 => 0
level14 => 0
level15 => 0
level16 => 0
level17 => 0
level18 => 0
level19 => 0
level20 => 0
level21 => 0
level22 => 0
level23 => 0
level24 => 0
level25 => 0
level26 => 0
level27 => 0
level28 => 0
level29 => 0
level30 => 0
level31 => 0
level32 => 0

Took 14.358724869s to build 50000000 items, 3.4822035e+06 items/sec
Took 725.695362ms to iterate 50000000 items
--- PASS: TestBuilder (15.08s)
PASS
ok  	github.com/couchbase/indexing/secondary/memdb/skiplist	57.273s
=== RUN   TestInsert
--- PASS: TestInsert (0.02s)
=== RUN   TestInsertPerf
20000000 items took 52.111887289s -> 383789.59274848 items/s snapshots_created 210 live_snapshots 1
--- PASS: TestInsertPerf (69.26s)
=== RUN   TestInsertDuplicates
--- PASS: TestInsertDuplicates (0.02s)
=== RUN   TestGetPerf
16000000 items took 23.004062996s -> 695529.3072698558 items/s
--- PASS: TestGetPerf (26.19s)
=== RUN   TestLoadStoreDisk
Inserting 1000000 items took 1.708477432s
node_count             = 1001382
soft_deletes           = 0
read_conflicts         = 0
insert_conflicts       = 3
next_pointers_per_node = NaN
memory_used            = 57348944
node_allocs            = 1000000
node_frees             = 0

level_node_distribution:
level0 => 750710
level1 => 187795
level2 => 47223
level3 => 11828
level4 => 2864
level5 => 716
level6 => 184
level7 => 53
level8 => 8
level9 => 1
level10 => 0
level11 => 0
level12 => 0
level13 => 0
level14 => 0
level15 => 0
level16 => 0
level17 => 0
level18 => 0
level19 => 0
level20 => 0
level21 => 0
level22 => 0
level23 => 0
level24 => 0
level25 => 0
level26 => 0
level27 => 0
level28 => 0
level29 => 0
level30 => 0
level31 => 0
level32 => 0

Storing to disk took 194.050576ms
Loading from disk took 520.796482ms
node_count             = 1000000
soft_deletes           = 0
read_conflicts         = 0
insert_conflicts       = 0
next_pointers_per_node = 1.3329
memory_used            = 57326928
node_allocs            = 1000000
node_frees             = 0

level_node_distribution:
level0 => 750419
level1 => 187158
level2 => 46803
level3 => 11680
level4 => 2928
level5 => 743
level6 => 200
level7 => 53
level8 => 13
level9 => 3
level10 => 0
level11 => 0
level12 => 0
level13 => 0
level14 => 0
level15 => 0
level16 => 0
level17 => 0
level18 => 0
level19 => 0
level20 => 0
level21 => 0
level22 => 0
level23 => 0
level24 => 0
level25 => 0
level26 => 0
level27 => 0
level28 => 0
level29 => 0
level30 => 0
level31 => 0
level32 => 0

--- PASS: TestLoadStoreDisk (3.92s)
=== RUN   TestStoreDiskShutdown
Inserting 1000000 items took 1.717781027s
node_count             = 1000002
soft_deletes           = 0
read_conflicts         = 0
insert_conflicts       = 12
next_pointers_per_node = NaN
memory_used            = 57327392
node_allocs            = 1000000
node_frees             = 0

level_node_distribution:
level0 => 750108
level1 => 187489
level2 => 46921
level3 => 11608
level4 => 2923
level5 => 699
level6 => 185
level7 => 49
level8 => 14
level9 => 5
level10 => 1
level11 => 0
level12 => 0
level13 => 0
level14 => 0
level15 => 0
level16 => 0
level17 => 0
level18 => 0
level19 => 0
level20 => 0
level21 => 0
level22 => 0
level23 => 0
level24 => 0
level25 => 0
level26 => 0
level27 => 0
level28 => 0
level29 => 0
level30 => 0
level31 => 0
level32 => 0

--- PASS: TestStoreDiskShutdown (2.55s)
=== RUN   TestDelete
node_count             = 10
soft_deletes           = 0
read_conflicts         = 0
insert_conflicts       = 0
next_pointers_per_node = NaN
memory_used            = 572
node_allocs            = 10
node_frees             = 0

level_node_distribution:
level0 => 8
level1 => 2
level2 => 0
level3 => 0
level4 => 0
level5 => 0
level6 => 0
level7 => 0
level8 => 0
level9 => 0
level10 => 0
level11 => 0
level12 => 0
level13 => 0
level14 => 0
level15 => 0
level16 => 0
level17 => 0
level18 => 0
level19 => 0
level20 => 0
level21 => 0
level22 => 0
level23 => 0
level24 => 0
level25 => 0
level26 => 0
level27 => 0
level28 => 0
level29 => 0
level30 => 0
level31 => 0
level32 => 0

node_count             = 10
soft_deletes           = 0
read_conflicts         = 0
insert_conflicts       = 0
next_pointers_per_node = NaN
memory_used            = 588
node_allocs            = 20
node_frees             = 10

level_node_distribution:
level0 => 7
level1 => 3
level2 => 0
level3 => 0
level4 => 0
level5 => 0
level6 => 0
level7 => 0
level8 => 0
level9 => 0
level10 => 0
level11 => 0
level12 => 0
level13 => 0
level14 => 0
level15 => 0
level16 => 0
level17 => 0
level18 => 0
level19 => 0
level20 => 0
level21 => 0
level22 => 0
level23 => 0
level24 => 0
level25 => 0
level26 => 0
level27 => 0
level28 => 0
level29 => 0
level30 => 0
level31 => 0
level32 => 0

--- PASS: TestDelete (1.00s)
=== RUN   TestGCPerf
final_node_count = 16000, average_live_node_count = 68459, wait_time_for_collection = 39ms
--- PASS: TestGCPerf (66.33s)
=== RUN   TestMemoryInUse
ItemsCount: 5000, MemoryInUse: 296864, NodesCount: 5000
ItemsCount: 0, MemoryInUse: 80, NodesCount: 0
--- PASS: TestMemoryInUse (1.02s)
=== RUN   TestFullScan
Inserting 1000000 items took 1.766696488s
node_count             = 1000000
soft_deletes           = 0
read_conflicts         = 0
insert_conflicts       = 8
next_pointers_per_node = NaN
memory_used            = 57326224
node_allocs            = 1000000
node_frees             = 0

level_node_distribution:
level0 => 750357
level1 => 187071
level2 => 46962
level3 => 11760
level4 => 2918
level5 => 723
level6 => 152
level7 => 44
level8 => 10
level9 => 3
level10 => 0
level11 => 0
level12 => 0
level13 => 0
level14 => 0
level15 => 0
level16 => 0
level17 => 0
level18 => 0
level19 => 0
level20 => 0
level21 => 0
level22 => 0
level23 => 0
level24 => 0
level25 => 0
level26 => 0
level27 => 0
level28 => 0
level29 => 0
level30 => 0
level31 => 0
level32 => 0

Full iteration of 1000000 items took 142.17046ms
--- PASS: TestFullScan (2.86s)
=== RUN   TestVisitor
node_count             = 1000000
soft_deletes           = 0
read_conflicts         = 0
insert_conflicts       = 0
next_pointers_per_node = NaN
memory_used            = 57337584
node_allocs            = 1000000
node_frees             = 0

level_node_distribution:
level0 => 749721
level1 => 187633
level2 => 47104
level3 => 11664
level4 => 2925
level5 => 730
level6 => 162
level7 => 47
level8 => 11
level9 => 3
level10 => 0
level11 => 0
level12 => 0
level13 => 0
level14 => 0
level15 => 0
level16 => 0
level17 => 0
level18 => 0
level19 => 0
level20 => 0
level21 => 0
level22 => 0
level23 => 0
level24 => 0
level25 => 0
level26 => 0
level27 => 0
level28 => 0
level29 => 0
level30 => 0
level31 => 0
level32 => 0

Took 85.103457ms to iterate 1000000 items, 1.1750404e+07 items/s
shard - 0 count = 10256, range: 0-10255
shard - 1 count = 10014, range: 10256-20269
shard - 2 count = 41971, range: 20270-62240
shard - 3 count = 100397, range: 62241-162637
shard - 4 count = 27595, range: 162638-190232
shard - 5 count = 15499, range: 190233-205731
shard - 6 count = 22373, range: 205732-228104
shard - 7 count = 37328, range: 228105-265432
shard - 8 count = 26329, range: 265433-291761
shard - 9 count = 38901, range: 291762-330662
shard - 10 count = 39759, range: 330663-370421
shard - 11 count = 21623, range: 370422-392044
shard - 12 count = 20589, range: 392045-412633
shard - 13 count = 45354, range: 412634-457987
shard - 14 count = 32520, range: 457988-490507
shard - 15 count = 16497, range: 490508-507004
shard - 16 count = 23085, range: 507005-530089
shard - 17 count = 32845, range: 530090-562934
shard - 18 count = 22237, range: 562935-585171
shard - 19 count = 53506, range: 585172-638677
shard - 20 count = 21512, range: 638678-660189
shard - 21 count = 55266, range: 660190-715455
shard - 22 count = 21975, range: 715456-737430
shard - 23 count = 5815, range: 737431-743245
shard - 24 count = 46865, range: 743246-790110
shard - 25 count = 86356, range: 790111-876466
shard - 26 count = 33675, range: 876467-910141
shard - 27 count = 18412, range: 910142-928553
shard - 28 count = 20480, range: 928554-949033
shard - 29 count = 16338, range: 949034-965371
shard - 30 count = 22857, range: 965372-988228
shard - 31 count = 11771, range: 988229-999999
--- PASS: TestVisitor (3.14s)
=== RUN   TestVisitorError
--- PASS: TestVisitorError (0.29s)
=== RUN   TestLoadDeltaStoreDisk
Storing to disk took 33.406167052s
Loading from disk took 2.21688605s
node_count             = 1000000
soft_deletes           = 0
read_conflicts         = 0
insert_conflicts       = 0
next_pointers_per_node = 1.3335
memory_used            = 57336064
node_allocs            = 1000000
node_frees             = 0

level_node_distribution:
level0 => 749886
level1 => 187711
level2 => 46767
level3 => 11635
level4 => 2991
level5 => 734
level6 => 231
level7 => 31
level8 => 10
level9 => 3
level10 => 1
level11 => 0
level12 => 0
level13 => 0
level14 => 0
level15 => 0
level16 => 0
level17 => 0
level18 => 0
level19 => 0
level20 => 0
level21 => 0
level22 => 0
level23 => 0
level24 => 0
level25 => 0
level26 => 0
level27 => 0
level28 => 0
level29 => 0
level30 => 0
level31 => 0
level32 => 0

Restored 26216
RestoredFailed 763929
--- PASS: TestLoadDeltaStoreDisk (39.09s)
=== RUN   TestNodeList
--- PASS: TestNodeList (0.00s)
PASS
ok  	github.com/couchbase/indexing/secondary/memdb	215.932s
=== RUN   TestForestDBIterator
--- PASS: TestForestDBIterator (0.01s)
=== RUN   TestForestDBIteratorSeek
--- PASS: TestForestDBIteratorSeek (0.00s)
=== RUN   TestPrimaryIndexEntry
--- PASS: TestPrimaryIndexEntry (0.00s)
=== RUN   TestSecondaryIndexEntry
--- PASS: TestSecondaryIndexEntry (0.00s)
=== RUN   TestPrimaryIndexEntryMatch
--- PASS: TestPrimaryIndexEntryMatch (0.00s)
=== RUN   TestSecondaryIndexEntryMatch
--- PASS: TestSecondaryIndexEntryMatch (0.00s)
=== RUN   TestMemDBInsertionPerf
2016-04-06T08:17:50.306+05:30 [Info] MemDBSlice:NewMemDBSlice Created New Slice Id 0 IndexInstId 0 WriterThreads 16
2016-04-06T08:17:50.613+05:30 [Info] logWriterStat:: 0 FlushedCount 1 QueuedCount 3124
2016-04-06T08:17:50.857+05:30 [Info] logWriterStat:: 0 FlushedCount 10000 QueuedCount 41636
2016-04-06T08:17:50.935+05:30 [Info] logWriterStat:: 0 FlushedCount 20000 QueuedCount 41041
2016-04-06T08:17:51.029+05:30 [Info] logWriterStat:: 0 FlushedCount 30000 QueuedCount 37844
2016-04-06T08:17:51.13+05:30 [Info] logWriterStat:: 0 FlushedCount 40000 QueuedCount 33503
2016-04-06T08:17:51.23+05:30 [Info] logWriterStat:: 0 FlushedCount 50000 QueuedCount 30591
2016-04-06T08:17:51.331+05:30 [Info] logWriterStat:: 0 FlushedCount 60000 QueuedCount 26875
2016-04-06T08:17:51.471+05:30 [Info] logWriterStat:: 0 FlushedCount 70000 QueuedCount 25503
2016-04-06T08:17:51.779+05:30 [Info] logWriterStat:: 0 FlushedCount 80000 QueuedCount 24869
2016-04-06T08:17:51.942+05:30 [Info] logWriterStat:: 0 FlushedCount 90000 QueuedCount 23597
2016-04-06T08:17:52.085+05:30 [Info] logWriterStat:: 0 FlushedCount 100000 QueuedCount 19505
2016-04-06T08:17:52.259+05:30 [Info] logWriterStat:: 0 FlushedCount 110000 QueuedCount 12704
2016-04-06T08:17:52.581+05:30 [Info] logWriterStat:: 0 FlushedCount 120000 QueuedCount 4302
2016-04-06T08:17:52.898+05:30 [Info] MemDBSlice::OpenSnapshot SliceId 0 IndexInstId 0 Creating New Snapshot SnapshotInfo: count:124505 committed:false
2016-04-06T08:17:53.047+05:30 [Info] logWriterStat:: 0 FlushedCount 130000 QueuedCount 21113
2016-04-06T08:17:53.367+05:30 [Info] logWriterStat:: 0 FlushedCount 140000 QueuedCount 49366
2016-04-06T08:17:53.585+05:30 [Info] logWriterStat:: 0 FlushedCount 150000 QueuedCount 49687
2016-04-06T08:17:53.714+05:30 [Info] logWriterStat:: 0 FlushedCount 160000 QueuedCount 48694
2016-04-06T08:17:53.813+05:30 [Info] logWriterStat:: 0 FlushedCount 170000 QueuedCount 48182
2016-04-06T08:17:53.921+05:30 [Info] logWriterStat:: 0 FlushedCount 180000 QueuedCount 48710
2016-04-06T08:17:54.05+05:30 [Info] logWriterStat:: 0 FlushedCount 190000 QueuedCount 48807
2016-04-06T08:17:54.161+05:30 [Info] logWriterStat:: 0 FlushedCount 200000 QueuedCount 48268
2016-04-06T08:17:54.26+05:30 [Info] logWriterStat:: 0 FlushedCount 210000 QueuedCount 47976
2016-04-06T08:17:54.368+05:30 [Info] logWriterStat:: 0 FlushedCount 220000 QueuedCount 46766
2016-04-06T08:17:54.48+05:30 [Info] logWriterStat:: 0 FlushedCount 230000 QueuedCount 47973
2016-04-06T08:17:54.583+05:30 [Info] logWriterStat:: 0 FlushedCount 240000 QueuedCount 47949
2016-04-06T08:17:54.695+05:30 [Info] logWriterStat:: 0 FlushedCount 250000 QueuedCount 48558
2016-04-06T08:17:54.792+05:30 [Info] logWriterStat:: 0 FlushedCount 260000 QueuedCount 47249
2016-04-06T08:17:54.895+05:30 [Info] logWriterStat:: 0 FlushedCount 270000 QueuedCount 47826
2016-04-06T08:17:54.999+05:30 [Info] logWriterStat:: 0 FlushedCount 280000 QueuedCount 48703
2016-04-06T08:17:55.097+05:30 [Info] logWriterStat:: 0 FlushedCount 290000 QueuedCount 47457
2016-04-06T08:17:55.202+05:30 [Info] logWriterStat:: 0 FlushedCount 300000 QueuedCount 47982
2016-04-06T08:17:55.309+05:30 [Info] logWriterStat:: 0 FlushedCount 310000 QueuedCount 48154
2016-04-06T08:17:55.413+05:30 [Info] logWriterStat:: 0 FlushedCount 320000 QueuedCount 48357
2016-04-06T08:17:55.536+05:30 [Info] logWriterStat:: 0 FlushedCount 330000 QueuedCount 48304
2016-04-06T08:17:55.739+05:30 [Info] logWriterStat:: 0 FlushedCount 340000 QueuedCount 49103
2016-04-06T08:17:56.086+05:30 [Info] logWriterStat:: 0 FlushedCount 350000 QueuedCount 48461
2016-04-06T08:17:56.209+05:30 [Info] logWriterStat:: 0 FlushedCount 360000 QueuedCount 41559
2016-04-06T08:17:56.317+05:30 [Info] logWriterStat:: 0 FlushedCount 370000 QueuedCount 40245
2016-04-06T08:17:56.485+05:30 [Info] logWriterStat:: 0 FlushedCount 380000 QueuedCount 49688
2016-04-06T08:17:56.724+05:30 [Info] logWriterStat:: 0 FlushedCount 390000 QueuedCount 49875
2016-04-06T08:17:56.923+05:30 [Info] logWriterStat:: 0 FlushedCount 400000 QueuedCount 48518
2016-04-06T08:17:57.02+05:30 [Info] logWriterStat:: 0 FlushedCount 410000 QueuedCount 48223
2016-04-06T08:17:57.124+05:30 [Info] logWriterStat:: 0 FlushedCount 420000 QueuedCount 48434
2016-04-06T08:17:57.217+05:30 [Info] logWriterStat:: 0 FlushedCount 430000 QueuedCount 47813
2016-04-06T08:17:57.322+05:30 [Info] logWriterStat:: 0 FlushedCount 440000 QueuedCount 47985
2016-04-06T08:17:57.425+05:30 [Info] logWriterStat:: 0 FlushedCount 450000 QueuedCount 48679
2016-04-06T08:17:57.539+05:30 [Info] logWriterStat:: 0 FlushedCount 460000 QueuedCount 47186
2016-04-06T08:17:57.65+05:30 [Info] logWriterStat:: 0 FlushedCount 470000 QueuedCount 48687
2016-04-06T08:17:57.755+05:30 [Info] logWriterStat:: 0 FlushedCount 480000 QueuedCount 48410
2016-04-06T08:17:57.861+05:30 [Info] logWriterStat:: 0 FlushedCount 490000 QueuedCount 48576
2016-04-06T08:17:57.974+05:30 [Info] logWriterStat:: 0 FlushedCount 500000 QueuedCount 48654
2016-04-06T08:17:58.08+05:30 [Info] logWriterStat:: 0 FlushedCount 510000 QueuedCount 48881
2016-04-06T08:17:58.192+05:30 [Info] logWriterStat:: 0 FlushedCount 520000 QueuedCount 48753
2016-04-06T08:17:58.295+05:30 [Info] logWriterStat:: 0 FlushedCount 530000 QueuedCount 48375
2016-04-06T08:17:58.408+05:30 [Info] logWriterStat:: 0 FlushedCount 540000 QueuedCount 48415
2016-04-06T08:17:58.516+05:30 [Info] logWriterStat:: 0 FlushedCount 550000 QueuedCount 48880
2016-04-06T08:17:58.618+05:30 [Info] logWriterStat:: 0 FlushedCount 560000 QueuedCount 48032
2016-04-06T08:17:58.839+05:30 [Info] logWriterStat:: 0 FlushedCount 570000 QueuedCount 49987
2016-04-06T08:17:59.218+05:30 [Info] logWriterStat:: 0 FlushedCount 580000 QueuedCount 48300
2016-04-06T08:17:59.491+05:30 [Info] logWriterStat:: 0 FlushedCount 590000 QueuedCount 49400
2016-04-06T08:17:59.673+05:30 [Info] logWriterStat:: 0 FlushedCount 600000 QueuedCount 47136
2016-04-06T08:17:59.831+05:30 [Info] logWriterStat:: 0 FlushedCount 610000 QueuedCount 48730
2016-04-06T08:17:59.947+05:30 [Info] logWriterStat:: 0 FlushedCount 620000 QueuedCount 45292
2016-04-06T08:18:00.073+05:30 [Info] logWriterStat:: 0 FlushedCount 630000 QueuedCount 48800
2016-04-06T08:18:00.235+05:30 [Info] logWriterStat:: 0 FlushedCount 640000 QueuedCount 48750
2016-04-06T08:18:00.327+05:30 [Info] logWriterStat:: 0 FlushedCount 650000 QueuedCount 44724
2016-04-06T08:18:00.47+05:30 [Info] logWriterStat:: 0 FlushedCount 660000 QueuedCount 49049
2016-04-06T08:18:00.604+05:30 [Info] logWriterStat:: 0 FlushedCount 670000 QueuedCount 48994
2016-04-06T08:18:00.72+05:30 [Info] logWriterStat:: 0 FlushedCount 680000 QueuedCount 48092
2016-04-06T08:18:00.827+05:30 [Info] logWriterStat:: 0 FlushedCount 690000 QueuedCount 48307
2016-04-06T08:18:00.931+05:30 [Info] logWriterStat:: 0 FlushedCount 700000 QueuedCount 48174
2016-04-06T08:18:01.035+05:30 [Info] logWriterStat:: 0 FlushedCount 710000 QueuedCount 48695
2016-04-06T08:18:01.139+05:30 [Info] logWriterStat:: 0 FlushedCount 720000 QueuedCount 47594
2016-04-06T08:18:01.251+05:30 [Info] logWriterStat:: 0 FlushedCount 730000 QueuedCount 48580
2016-04-06T08:18:01.361+05:30 [Info] logWriterStat:: 0 FlushedCount 740000 QueuedCount 48332
2016-04-06T08:18:01.483+05:30 [Info] logWriterStat:: 0 FlushedCount 750000 QueuedCount 48333
2016-04-06T08:18:01.588+05:30 [Info] logWriterStat:: 0 FlushedCount 760000 QueuedCount 47200
2016-04-06T08:18:01.695+05:30 [Info] logWriterStat:: 0 FlushedCount 770000 QueuedCount 47855
2016-04-06T08:18:01.816+05:30 [Info] logWriterStat:: 0 FlushedCount 780000 QueuedCount 48531
2016-04-06T08:18:01.925+05:30 [Info] logWriterStat:: 0 FlushedCount 790000 QueuedCount 48189
2016-04-06T08:18:02.025+05:30 [Info] logWriterStat:: 0 FlushedCount 800000 QueuedCount 47021
2016-04-06T08:18:02.176+05:30 [Info] logWriterStat:: 0 FlushedCount 810000 QueuedCount 49146
2016-04-06T08:18:02.297+05:30 [Info] logWriterStat:: 0 FlushedCount 820000 QueuedCount 48883
2016-04-06T08:18:02.412+05:30 [Info] logWriterStat:: 0 FlushedCount 830000 QueuedCount 48774
2016-04-06T08:18:02.531+05:30 [Info] logWriterStat:: 0 FlushedCount 840000 QueuedCount 48771
2016-04-06T08:18:02.642+05:30 [Info] logWriterStat:: 0 FlushedCount 850000 QueuedCount 48880
2016-04-06T08:18:02.756+05:30 [Info] logWriterStat:: 0 FlushedCount 860000 QueuedCount 48945
2016-04-06T08:18:02.856+05:30 [Info] logWriterStat:: 0 FlushedCount 870000 QueuedCount 49304
2016-04-06T08:18:02.959+05:30 [Info] logWriterStat:: 0 FlushedCount 880000 QueuedCount 48740
2016-04-06T08:18:03.068+05:30 [Info] logWriterStat:: 0 FlushedCount 890000 QueuedCount 47708
2016-04-06T08:18:03.17+05:30 [Info] logWriterStat:: 0 FlushedCount 900000 QueuedCount 48463
2016-04-06T08:18:03.276+05:30 [Info] logWriterStat:: 0 FlushedCount 910000 QueuedCount 48274
2016-04-06T08:18:03.611+05:30 [Info] logWriterStat:: 0 FlushedCount 920000 QueuedCount 48915
2016-04-06T08:18:04.004+05:30 [Info] logWriterStat:: 0 FlushedCount 930000 QueuedCount 49750
2016-04-06T08:18:04.317+05:30 [Info] logWriterStat:: 0 FlushedCount 940000 QueuedCount 44649
2016-04-06T08:18:04.434+05:30 [Info] logWriterStat:: 0 FlushedCount 950000 QueuedCount 40229
2016-04-06T08:18:04.575+05:30 [Info] logWriterStat:: 0 FlushedCount 960000 QueuedCount 43148
2016-04-06T08:18:04.872+05:30 [Info] logWriterStat:: 0 FlushedCount 970000 QueuedCount 49736
2016-04-06T08:18:05.22+05:30 [Info] logWriterStat:: 0 FlushedCount 980000 QueuedCount 49489
2016-04-06T08:18:05.476+05:30 [Info] logWriterStat:: 0 FlushedCount 990000 QueuedCount 49971
2016-04-06T08:18:05.648+05:30 [Info] logWriterStat:: 0 FlushedCount 1000000 QueuedCount 48593
2016-04-06T08:18:05.796+05:30 [Info] logWriterStat:: 0 FlushedCount 1010000 QueuedCount 48120
2016-04-06T08:18:05.903+05:30 [Info] logWriterStat:: 0 FlushedCount 1020000 QueuedCount 48103
2016-04-06T08:18:06.069+05:30 [Info] logWriterStat:: 0 FlushedCount 1030000 QueuedCount 48994
2016-04-06T08:18:06.183+05:30 [Info] logWriterStat:: 0 FlushedCount 1040000 QueuedCount 48095
2016-04-06T08:18:06.29+05:30 [Info] logWriterStat:: 0 FlushedCount 1050000 QueuedCount 47688
2016-04-06T08:18:06.397+05:30 [Info] logWriterStat:: 0 FlushedCount 1060000 QueuedCount 48480
2016-04-06T08:18:06.516+05:30 [Info] logWriterStat:: 0 FlushedCount 1070000 QueuedCount 49037
2016-04-06T08:18:06.615+05:30 [Info] logWriterStat:: 0 FlushedCount 1080000 QueuedCount 46997
2016-04-06T08:18:06.73+05:30 [Info] logWriterStat:: 0 FlushedCount 1090000 QueuedCount 48947
2016-04-06T08:18:06.851+05:30 [Info] logWriterStat:: 0 FlushedCount 1100000 QueuedCount 48678
2016-04-06T08:18:06.96+05:30 [Info] logWriterStat:: 0 FlushedCount 1110000 QueuedCount 48589
2016-04-06T08:18:07.067+05:30 [Info] logWriterStat:: 0 FlushedCount 1120000 QueuedCount 47993
2016-04-06T08:18:07.169+05:30 [Info] logWriterStat:: 0 FlushedCount 1130000 QueuedCount 48511
2016-04-06T08:18:07.275+05:30 [Info] logWriterStat:: 0 FlushedCount 1140000 QueuedCount 48150
2016-04-06T08:18:07.39+05:30 [Info] logWriterStat:: 0 FlushedCount 1150000 QueuedCount 47808
2016-04-06T08:18:07.5+05:30 [Info] logWriterStat:: 0 FlushedCount 1160000 QueuedCount 48276
2016-04-06T08:18:07.604+05:30 [Info] logWriterStat:: 0 FlushedCount 1170000 QueuedCount 48369
2016-04-06T08:18:07.708+05:30 [Info] logWriterStat:: 0 FlushedCount 1180000 QueuedCount 47365
2016-04-06T08:18:07.818+05:30 [Info] logWriterStat:: 0 FlushedCount 1190000 QueuedCount 47389
2016-04-06T08:18:07.932+05:30 [Info] logWriterStat:: 0 FlushedCount 1200000 QueuedCount 49139
2016-04-06T08:18:08.037+05:30 [Info] logWriterStat:: 0 FlushedCount 1210000 QueuedCount 47992
2016-04-06T08:18:08.149+05:30 [Info] logWriterStat:: 0 FlushedCount 1220000 QueuedCount 47470
2016-04-06T08:18:08.248+05:30 [Info] logWriterStat:: 0 FlushedCount 1230000 QueuedCount 49059
2016-04-06T08:18:08.351+05:30 [Info] logWriterStat:: 0 FlushedCount 1240000 QueuedCount 48853
2016-04-06T08:18:08.47+05:30 [Info] logWriterStat:: 0 FlushedCount 1250000 QueuedCount 48911
2016-04-06T08:18:08.578+05:30 [Info] logWriterStat:: 0 FlushedCount 1260000 QueuedCount 48964
2016-04-06T08:18:08.707+05:30 [Info] logWriterStat:: 0 FlushedCount 1270000 QueuedCount 48520
2016-04-06T08:18:08.807+05:30 [Info] logWriterStat:: 0 FlushedCount 1280000 QueuedCount 47679
2016-04-06T08:18:08.913+05:30 [Info] logWriterStat:: 0 FlushedCount 1290000 QueuedCount 48430
2016-04-06T08:18:09.024+05:30 [Info] logWriterStat:: 0 FlushedCount 1300000 QueuedCount 48271
2016-04-06T08:18:09.147+05:30 [Info] logWriterStat:: 0 FlushedCount 1310000 QueuedCount 48209
2016-04-06T08:18:09.251+05:30 [Info] logWriterStat:: 0 FlushedCount 1320000 QueuedCount 48824
2016-04-06T08:18:09.36+05:30 [Info] logWriterStat:: 0 FlushedCount 1330000 QueuedCount 48693
2016-04-06T08:18:09.479+05:30 [Info] logWriterStat:: 0 FlushedCount 1340000 QueuedCount 47875
2016-04-06T08:18:09.628+05:30 [Info] logWriterStat:: 0 FlushedCount 1350000 QueuedCount 49810
2016-04-06T08:18:09.983+05:30 [Info] logWriterStat:: 0 FlushedCount 1360000 QueuedCount 48053
2016-04-06T08:18:10.288+05:30 [Info] logWriterStat:: 0 FlushedCount 1370000 QueuedCount 47153
2016-04-06T08:18:10.687+05:30 [Info] logWriterStat:: 0 FlushedCount 1380000 QueuedCount 46936
2016-04-06T08:18:10.969+05:30 [Info] logWriterStat:: 0 FlushedCount 1390000 QueuedCount 48579
2016-04-06T08:18:11.188+05:30 [Info] logWriterStat:: 0 FlushedCount 1400000 QueuedCount 48773
2016-04-06T08:18:11.346+05:30 [Info] logWriterStat:: 0 FlushedCount 1410000 QueuedCount 49959
2016-04-06T08:18:11.465+05:30 [Info] logWriterStat:: 0 FlushedCount 1420000 QueuedCount 48026
2016-04-06T08:18:11.585+05:30 [Info] logWriterStat:: 0 FlushedCount 1430000 QueuedCount 47093
2016-04-06T08:18:11.697+05:30 [Info] logWriterStat:: 0 FlushedCount 1440000 QueuedCount 46784
2016-04-06T08:18:11.856+05:30 [Info] logWriterStat:: 0 FlushedCount 1450000 QueuedCount 47740
2016-04-06T08:18:11.958+05:30 [Info] logWriterStat:: 0 FlushedCount 1460000 QueuedCount 43189
2016-04-06T08:18:12.128+05:30 [Info] logWriterStat:: 0 FlushedCount 1470000 QueuedCount 45248
2016-04-06T08:18:12.24+05:30 [Info] logWriterStat:: 0 FlushedCount 1480000 QueuedCount 44492
2016-04-06T08:18:12.313+05:30 [Info] logWriterStat:: 0 FlushedCount 1490000 QueuedCount 43229
2016-04-06T08:18:12.382+05:30 [Info] logWriterStat:: 0 FlushedCount 1500000 QueuedCount 41907
2016-04-06T08:18:12.464+05:30 [Info] logWriterStat:: 0 FlushedCount 1510000 QueuedCount 39871
2016-04-06T08:18:12.539+05:30 [Info] logWriterStat:: 0 FlushedCount 1520000 QueuedCount 37820
2016-04-06T08:18:12.617+05:30 [Info] logWriterStat:: 0 FlushedCount 1530000 QueuedCount 36215
2016-04-06T08:18:12.699+05:30 [Info] logWriterStat:: 0 FlushedCount 1540000 QueuedCount 35276
2016-04-06T08:18:12.781+05:30 [Info] logWriterStat:: 0 FlushedCount 1550000 QueuedCount 34468
2016-04-06T08:18:12.86+05:30 [Info] logWriterStat:: 0 FlushedCount 1560000 QueuedCount 34357
2016-04-06T08:18:12.935+05:30 [Info] logWriterStat:: 0 FlushedCount 1570000 QueuedCount 34008
2016-04-06T08:18:13.016+05:30 [Info] logWriterStat:: 0 FlushedCount 1580000 QueuedCount 33032
2016-04-06T08:18:13.091+05:30 [Info] logWriterStat:: 0 FlushedCount 1590000 QueuedCount 32101
2016-04-06T08:18:13.166+05:30 [Info] logWriterStat:: 0 FlushedCount 1600000 QueuedCount 31239
2016-04-06T08:18:13.245+05:30 [Info] logWriterStat:: 0 FlushedCount 1610000 QueuedCount 30696
2016-04-06T08:18:13.319+05:30 [Info] logWriterStat:: 0 FlushedCount 1620000 QueuedCount 29141
2016-04-06T08:18:13.391+05:30 [Info] logWriterStat:: 0 FlushedCount 1630000 QueuedCount 27311
2016-04-06T08:18:13.474+05:30 [Info] logWriterStat:: 0 FlushedCount 1640000 QueuedCount 25170
2016-04-06T08:18:13.549+05:30 [Info] logWriterStat:: 0 FlushedCount 1650000 QueuedCount 22908
2016-04-06T08:18:13.618+05:30 [Info] logWriterStat:: 0 FlushedCount 1660000 QueuedCount 19946
2016-04-06T08:18:13.686+05:30 [Info] logWriterStat:: 0 FlushedCount 1670000 QueuedCount 15856
2016-04-06T08:18:13.757+05:30 [Info] logWriterStat:: 0 FlushedCount 1680000 QueuedCount 12446
2016-04-06T08:18:13.818+05:30 [Info] logWriterStat:: 0 FlushedCount 1690000 QueuedCount 8461
2016-04-06T08:18:13.912+05:30 [Info] logWriterStat:: 0 FlushedCount 1700000 QueuedCount 1738
2016-04-06T08:18:13.939+05:30 [Info] MemDBSlice::OpenSnapshot SliceId 0 IndexInstId 0 Creating New Snapshot SnapshotInfo: count:1701738 committed:false
2016-04-06T08:18:14.063+05:30 [Info] logWriterStat:: 0 FlushedCount 1710000 QueuedCount 49900
2016-04-06T08:18:14.156+05:30 [Info] logWriterStat:: 0 FlushedCount 1720000 QueuedCount 49838
2016-04-06T08:18:14.244+05:30 [Info] logWriterStat:: 0 FlushedCount 1730000 QueuedCount 49856
2016-04-06T08:18:14.333+05:30 [Info] logWriterStat:: 0 FlushedCount 1740000 QueuedCount 49995
2016-04-06T08:18:14.419+05:30 [Info] logWriterStat:: 0 FlushedCount 1750000 QueuedCount 49879
2016-04-06T08:18:14.516+05:30 [Info] logWriterStat:: 0 FlushedCount 1760000 QueuedCount 49846
2016-04-06T08:18:14.615+05:30 [Info] logWriterStat:: 0 FlushedCount 1770000 QueuedCount 49865
2016-04-06T08:18:14.704+05:30 [Info] logWriterStat:: 0 FlushedCount 1780000 QueuedCount 49913
2016-04-06T08:18:14.794+05:30 [Info] logWriterStat:: 0 FlushedCount 1790000 QueuedCount 49969
2016-04-06T08:18:14.884+05:30 [Info] logWriterStat:: 0 FlushedCount 1800000 QueuedCount 49891
2016-04-06T08:18:14.972+05:30 [Info] logWriterStat:: 0 FlushedCount 1810000 QueuedCount 49816
2016-04-06T08:18:15.061+05:30 [Info] logWriterStat:: 0 FlushedCount 1820000 QueuedCount 49958
2016-04-06T08:18:15.15+05:30 [Info] logWriterStat:: 0 FlushedCount 1830000 QueuedCount 49900
2016-04-06T08:18:15.24+05:30 [Info] logWriterStat:: 0 FlushedCount 1840000 QueuedCount 49922
2016-04-06T08:18:15.326+05:30 [Info] logWriterStat:: 0 FlushedCount 1850000 QueuedCount 49885
2016-04-06T08:18:15.413+05:30 [Info] logWriterStat:: 0 FlushedCount 1860000 QueuedCount 49892
2016-04-06T08:18:15.504+05:30 [Info] logWriterStat:: 0 FlushedCount 1870000 QueuedCount 49927
2016-04-06T08:18:15.593+05:30 [Info] logWriterStat:: 0 FlushedCount 1880000 QueuedCount 49969
2016-04-06T08:18:15.68+05:30 [Info] logWriterStat:: 0 FlushedCount 1890000 QueuedCount 49979
2016-04-06T08:18:15.787+05:30 [Info] logWriterStat:: 0 FlushedCount 1900000 QueuedCount 49800
2016-04-06T08:18:15.873+05:30 [Info] logWriterStat:: 0 FlushedCount 1910000 QueuedCount 49904
2016-04-06T08:18:15.959+05:30 [Info] logWriterStat:: 0 FlushedCount 1920000 QueuedCount 49925
2016-04-06T08:18:16.044+05:30 [Info] logWriterStat:: 0 FlushedCount 1930000 QueuedCount 49894
2016-04-06T08:18:16.128+05:30 [Info] logWriterStat:: 0 FlushedCount 1940000 QueuedCount 49932
2016-04-06T08:18:16.21+05:30 [Info] logWriterStat:: 0 FlushedCount 1950000 QueuedCount 49946
2016-04-06T08:18:16.295+05:30 [Info] logWriterStat:: 0 FlushedCount 1960000 QueuedCount 49943
2016-04-06T08:18:16.377+05:30 [Info] logWriterStat:: 0 FlushedCount 1970000 QueuedCount 49946
2016-04-06T08:18:16.468+05:30 [Info] logWriterStat:: 0 FlushedCount 1980000 QueuedCount 49882
2016-04-06T08:18:16.55+05:30 [Info] logWriterStat:: 0 FlushedCount 1990000 QueuedCount 49926
2016-04-06T08:18:16.634+05:30 [Info] logWriterStat:: 0 FlushedCount 2000000 QueuedCount 49895
2016-04-06T08:18:16.717+05:30 [Info] logWriterStat:: 0 FlushedCount 2010000 QueuedCount 49957
2016-04-06T08:18:16.805+05:30 [Info] logWriterStat:: 0 FlushedCount 2020000 QueuedCount 49817
2016-04-06T08:18:16.883+05:30 [Info] logWriterStat:: 0 FlushedCount 2030000 QueuedCount 48681
2016-04-06T08:18:16.985+05:30 [Info] logWriterStat:: 0 FlushedCount 2040000 QueuedCount 49938
2016-04-06T08:18:17.067+05:30 [Info] logWriterStat:: 0 FlushedCount 2050000 QueuedCount 49897
2016-04-06T08:18:17.148+05:30 [Info] logWriterStat:: 0 FlushedCount 2060000 QueuedCount 49972
2016-04-06T08:18:17.228+05:30 [Info] logWriterStat:: 0 FlushedCount 2070000 QueuedCount 49946
2016-04-06T08:18:17.312+05:30 [Info] logWriterStat:: 0 FlushedCount 2080000 QueuedCount 49983
2016-04-06T08:18:17.396+05:30 [Info] logWriterStat:: 0 FlushedCount 2090000 QueuedCount 49926
2016-04-06T08:18:17.487+05:30 [Info] logWriterStat:: 0 FlushedCount 2100000 QueuedCount 49935
2016-04-06T08:18:17.57+05:30 [Info] logWriterStat:: 0 FlushedCount 2110000 QueuedCount 49991
2016-04-06T08:18:17.653+05:30 [Info] logWriterStat:: 0 FlushedCount 2120000 QueuedCount 49990
2016-04-06T08:18:17.736+05:30 [Info] logWriterStat:: 0 FlushedCount 2130000 QueuedCount 49925
2016-04-06T08:18:17.815+05:30 [Info] logWriterStat:: 0 FlushedCount 2140000 QueuedCount 49934
2016-04-06T08:18:17.896+05:30 [Info] logWriterStat:: 0 FlushedCount 2150000 QueuedCount 49908
2016-04-06T08:18:17.979+05:30 [Info] logWriterStat:: 0 FlushedCount 2160000 QueuedCount 49965
2016-04-06T08:18:18.057+05:30 [Info] logWriterStat:: 0 FlushedCount 2170000 QueuedCount 49962
2016-04-06T08:18:18.14+05:30 [Info] logWriterStat:: 0 FlushedCount 2180000 QueuedCount 49948
2016-04-06T08:18:18.223+05:30 [Info] logWriterStat:: 0 FlushedCount 2190000 QueuedCount 49936
2016-04-06T08:18:18.303+05:30 [Info] logWriterStat:: 0 FlushedCount 2200000 QueuedCount 49988
2016-04-06T08:18:18.385+05:30 [Info] logWriterStat:: 0 FlushedCount 2210000 QueuedCount 49968
2016-04-06T08:18:18.477+05:30 [Info] logWriterStat:: 0 FlushedCount 2220000 QueuedCount 49938
2016-04-06T08:18:18.559+05:30 [Info] logWriterStat:: 0 FlushedCount 2230000 QueuedCount 49964
2016-04-06T08:18:18.637+05:30 [Info] logWriterStat:: 0 FlushedCount 2240000 QueuedCount 49916
2016-04-06T08:18:18.727+05:30 [Info] logWriterStat:: 0 FlushedCount 2250000 QueuedCount 49980
2016-04-06T08:18:18.805+05:30 [Info] logWriterStat:: 0 FlushedCount 2260000 QueuedCount 49889
2016-04-06T08:18:18.889+05:30 [Info] logWriterStat:: 0 FlushedCount 2270000 QueuedCount 49808
2016-04-06T08:18:18.974+05:30 [Info] logWriterStat:: 0 FlushedCount 2280000 QueuedCount 49897
2016-04-06T08:18:19.064+05:30 [Info] logWriterStat:: 0 FlushedCount 2290000 QueuedCount 49972
2016-04-06T08:18:19.143+05:30 [Info] logWriterStat:: 0 FlushedCount 2300000 QueuedCount 49970
2016-04-06T08:18:19.222+05:30 [Info] logWriterStat:: 0 FlushedCount 2310000 QueuedCount 49834
2016-04-06T08:18:19.302+05:30 [Info] logWriterStat:: 0 FlushedCount 2320000 QueuedCount 49910
2016-04-06T08:18:19.381+05:30 [Info] logWriterStat:: 0 FlushedCount 2330000 QueuedCount 49986
2016-04-06T08:18:19.469+05:30 [Info] logWriterStat:: 0 FlushedCount 2340000 QueuedCount 49933
2016-04-06T08:18:19.551+05:30 [Info] logWriterStat:: 0 FlushedCount 2350000 QueuedCount 49948
2016-04-06T08:18:19.63+05:30 [Info] logWriterStat:: 0 FlushedCount 2360000 QueuedCount 49921
2016-04-06T08:18:19.71+05:30 [Info] logWriterStat:: 0 FlushedCount 2370000 QueuedCount 49931
2016-04-06T08:18:19.797+05:30 [Info] logWriterStat:: 0 FlushedCount 2380000 QueuedCount 49919
2016-04-06T08:18:19.88+05:30 [Info] logWriterStat:: 0 FlushedCount 2390000 QueuedCount 49932
2016-04-06T08:18:19.963+05:30 [Info] logWriterStat:: 0 FlushedCount 2400000 QueuedCount 49869
2016-04-06T08:18:20.045+05:30 [Info] logWriterStat:: 0 FlushedCount 2410000 QueuedCount 49910
2016-04-06T08:18:20.123+05:30 [Info] logWriterStat:: 0 FlushedCount 2420000 QueuedCount 49933
2016-04-06T08:18:20.204+05:30 [Info] logWriterStat:: 0 FlushedCount 2430000 QueuedCount 49939
2016-04-06T08:18:20.283+05:30 [Info] logWriterStat:: 0 FlushedCount 2440000 QueuedCount 49918
2016-04-06T08:18:20.366+05:30 [Info] logWriterStat:: 0 FlushedCount 2450000 QueuedCount 49919
2016-04-06T08:18:20.468+05:30 [Info] logWriterStat:: 0 FlushedCount 2460000 QueuedCount 49006
2016-04-06T08:18:20.575+05:30 [Info] logWriterStat:: 0 FlushedCount 2470000 QueuedCount 47456
2016-04-06T08:18:20.65+05:30 [Info] logWriterStat:: 0 FlushedCount 2480000 QueuedCount 47748
2016-04-06T08:18:20.741+05:30 [Info] logWriterStat:: 0 FlushedCount 2490000 QueuedCount 47834
2016-04-06T08:18:20.823+05:30 [Info] logWriterStat:: 0 FlushedCount 2500000 QueuedCount 47764
2016-04-06T08:18:20.899+05:30 [Info] logWriterStat:: 0 FlushedCount 2510000 QueuedCount 46951
2016-04-06T08:18:20.984+05:30 [Info] logWriterStat:: 0 FlushedCount 2520000 QueuedCount 48235
2016-04-06T08:18:21.098+05:30 [Info] logWriterStat:: 0 FlushedCount 2530000 QueuedCount 48280
2016-04-06T08:18:21.177+05:30 [Info] logWriterStat:: 0 FlushedCount 2540000 QueuedCount 47551
2016-04-06T08:18:21.251+05:30 [Info] logWriterStat:: 0 FlushedCount 2550000 QueuedCount 48602
2016-04-06T08:18:21.327+05:30 [Info] logWriterStat:: 0 FlushedCount 2560000 QueuedCount 46842
2016-04-06T08:18:21.406+05:30 [Info] logWriterStat:: 0 FlushedCount 2570000 QueuedCount 47739
2016-04-06T08:18:21.493+05:30 [Info] logWriterStat:: 0 FlushedCount 2580000 QueuedCount 48521
2016-04-06T08:18:21.572+05:30 [Info] logWriterStat:: 0 FlushedCount 2590000 QueuedCount 47717
2016-04-06T08:18:21.65+05:30 [Info] logWriterStat:: 0 FlushedCount 2600000 QueuedCount 47816
2016-04-06T08:18:21.744+05:30 [Info] logWriterStat:: 0 FlushedCount 2610000 QueuedCount 48300
2016-04-06T08:18:21.83+05:30 [Info] logWriterStat:: 0 FlushedCount 2620000 QueuedCount 48439
2016-04-06T08:18:21.911+05:30 [Info] logWriterStat:: 0 FlushedCount 2630000 QueuedCount 47762
2016-04-06T08:18:22.006+05:30 [Info] logWriterStat:: 0 FlushedCount 2640000 QueuedCount 47785
2016-04-06T08:18:22.085+05:30 [Info] logWriterStat:: 0 FlushedCount 2650000 QueuedCount 47635
2016-04-06T08:18:22.168+05:30 [Info] logWriterStat:: 0 FlushedCount 2660000 QueuedCount 47021
2016-04-06T08:18:22.246+05:30 [Info] logWriterStat:: 0 FlushedCount 2670000 QueuedCount 47727
2016-04-06T08:18:22.327+05:30 [Info] logWriterStat:: 0 FlushedCount 2680000 QueuedCount 48048
2016-04-06T08:18:22.407+05:30 [Info] logWriterStat:: 0 FlushedCount 2690000 QueuedCount 47922
2016-04-06T08:18:22.501+05:30 [Info] logWriterStat:: 0 FlushedCount 2700000 QueuedCount 46557
2016-04-06T08:18:22.585+05:30 [Info] logWriterStat:: 0 FlushedCount 2710000 QueuedCount 47041
2016-04-06T08:18:22.664+05:30 [Info] logWriterStat:: 0 FlushedCount 2720000 QueuedCount 48115
2016-04-06T08:18:22.741+05:30 [Info] logWriterStat:: 0 FlushedCount 2730000 QueuedCount 48027
2016-04-06T08:18:22.821+05:30 [Info] logWriterStat:: 0 FlushedCount 2740000 QueuedCount 47342
2016-04-06T08:18:22.903+05:30 [Info] logWriterStat:: 0 FlushedCount 2750000 QueuedCount 47861
2016-04-06T08:18:22.984+05:30 [Info] logWriterStat:: 0 FlushedCount 2760000 QueuedCount 48355
2016-04-06T08:18:23.069+05:30 [Info] logWriterStat:: 0 FlushedCount 2770000 QueuedCount 47854
2016-04-06T08:18:23.149+05:30 [Info] logWriterStat:: 0 FlushedCount 2780000 QueuedCount 46989
2016-04-06T08:18:23.228+05:30 [Info] logWriterStat:: 0 FlushedCount 2790000 QueuedCount 47759
2016-04-06T08:18:23.307+05:30 [Info] logWriterStat:: 0 FlushedCount 2800000 QueuedCount 48059
2016-04-06T08:18:23.38+05:30 [Info] logWriterStat:: 0 FlushedCount 2810000 QueuedCount 48040
2016-04-06T08:18:23.465+05:30 [Info] logWriterStat:: 0 FlushedCount 2820000 QueuedCount 47951
2016-04-06T08:18:23.541+05:30 [Info] logWriterStat:: 0 FlushedCount 2830000 QueuedCount 47362
2016-04-06T08:18:23.619+05:30 [Info] logWriterStat:: 0 FlushedCount 2840000 QueuedCount 47455
2016-04-06T08:18:23.704+05:30 [Info] logWriterStat:: 0 FlushedCount 2850000 QueuedCount 47850
2016-04-06T08:18:23.787+05:30 [Info] logWriterStat:: 0 FlushedCount 2860000 QueuedCount 47291
2016-04-06T08:18:23.869+05:30 [Info] logWriterStat:: 0 FlushedCount 2870000 QueuedCount 47354
2016-04-06T08:18:23.959+05:30 [Info] logWriterStat:: 0 FlushedCount 2880000 QueuedCount 47623
2016-04-06T08:18:24.055+05:30 [Info] logWriterStat:: 0 FlushedCount 2890000 QueuedCount 49971
2016-04-06T08:18:24.137+05:30 [Info] logWriterStat:: 0 FlushedCount 2900000 QueuedCount 49979
2016-04-06T08:18:24.218+05:30 [Info] logWriterStat:: 0 FlushedCount 2910000 QueuedCount 49853
2016-04-06T08:18:24.297+05:30 [Info] logWriterStat:: 0 FlushedCount 2920000 QueuedCount 49925
2016-04-06T08:18:24.379+05:30 [Info] logWriterStat:: 0 FlushedCount 2930000 QueuedCount 49969
2016-04-06T08:18:24.471+05:30 [Info] logWriterStat:: 0 FlushedCount 2940000 QueuedCount 49956
2016-04-06T08:18:24.554+05:30 [Info] logWriterStat:: 0 FlushedCount 2950000 QueuedCount 49881
2016-04-06T08:18:24.64+05:30 [Info] logWriterStat:: 0 FlushedCount 2960000 QueuedCount 49946
2016-04-06T08:18:24.722+05:30 [Info] logWriterStat:: 0 FlushedCount 2970000 QueuedCount 49965
2016-04-06T08:18:24.805+05:30 [Info] logWriterStat:: 0 FlushedCount 2980000 QueuedCount 49928
2016-04-06T08:18:24.886+05:30 [Info] logWriterStat:: 0 FlushedCount 2990000 QueuedCount 49959
2016-04-06T08:18:24.969+05:30 [Info] logWriterStat:: 0 FlushedCount 3000000 QueuedCount 49963
2016-04-06T08:18:25.047+05:30 [Info] logWriterStat:: 0 FlushedCount 3010000 QueuedCount 49944
2016-04-06T08:18:25.126+05:30 [Info] logWriterStat:: 0 FlushedCount 3020000 QueuedCount 49945
2016-04-06T08:18:25.206+05:30 [Info] logWriterStat:: 0 FlushedCount 3030000 QueuedCount 49862
2016-04-06T08:18:25.282+05:30 [Info] logWriterStat:: 0 FlushedCount 3040000 QueuedCount 49894
2016-04-06T08:18:25.364+05:30 [Info] logWriterStat:: 0 FlushedCount 3050000 QueuedCount 49953
2016-04-06T08:18:25.458+05:30 [Info] logWriterStat:: 0 FlushedCount 3060000 QueuedCount 49920
2016-04-06T08:18:25.539+05:30 [Info] logWriterStat:: 0 FlushedCount 3070000 QueuedCount 49882
2016-04-06T08:18:25.62+05:30 [Info] logWriterStat:: 0 FlushedCount 3080000 QueuedCount 49923
2016-04-06T08:18:25.699+05:30 [Info] logWriterStat:: 0 FlushedCount 3090000 QueuedCount 49968
2016-04-06T08:18:25.793+05:30 [Info] logWriterStat:: 0 FlushedCount 3100000 QueuedCount 49953
2016-04-06T08:18:25.883+05:30 [Info] logWriterStat:: 0 FlushedCount 3110000 QueuedCount 49890
2016-04-06T08:18:25.966+05:30 [Info] logWriterStat:: 0 FlushedCount 3120000 QueuedCount 49974
2016-04-06T08:18:26.044+05:30 [Info] logWriterStat:: 0 FlushedCount 3130000 QueuedCount 49854
2016-04-06T08:18:26.131+05:30 [Info] logWriterStat:: 0 FlushedCount 3140000 QueuedCount 49955
2016-04-06T08:18:26.216+05:30 [Info] logWriterStat:: 0 FlushedCount 3150000 QueuedCount 49971
2016-04-06T08:18:26.304+05:30 [Info] logWriterStat:: 0 FlushedCount 3160000 QueuedCount 49926
2016-04-06T08:18:26.385+05:30 [Info] logWriterStat:: 0 FlushedCount 3170000 QueuedCount 49917
2016-04-06T08:18:26.481+05:30 [Info] logWriterStat:: 0 FlushedCount 3180000 QueuedCount 49912
2016-04-06T08:18:26.56+05:30 [Info] logWriterStat:: 0 FlushedCount 3190000 QueuedCount 49976
2016-04-06T08:18:26.642+05:30 [Info] logWriterStat:: 0 FlushedCount 3200000 QueuedCount 49310
2016-04-06T08:18:26.714+05:30 [Info] logWriterStat:: 0 FlushedCount 3210000 QueuedCount 47849
2016-04-06T08:18:26.797+05:30 [Info] logWriterStat:: 0 FlushedCount 3220000 QueuedCount 47818
2016-04-06T08:18:26.874+05:30 [Info] logWriterStat:: 0 FlushedCount 3230000 QueuedCount 49217
2016-04-06T08:18:26.955+05:30 [Info] logWriterStat:: 0 FlushedCount 3240000 QueuedCount 49047
2016-04-06T08:18:27.05+05:30 [Info] logWriterStat:: 0 FlushedCount 3250000 QueuedCount 48592
2016-04-06T08:18:27.827+05:30 [Info] logWriterStat:: 0 FlushedCount 3260000 QueuedCount 49940
2016-04-06T08:18:28.116+05:30 [Info] logWriterStat:: 0 FlushedCount 3270000 QueuedCount 49946
2016-04-06T08:18:28.265+05:30 [Info] logWriterStat:: 0 FlushedCount 3280000 QueuedCount 49854
2016-04-06T08:18:28.368+05:30 [Info] logWriterStat:: 0 FlushedCount 3290000 QueuedCount 49954
2016-04-06T08:18:28.518+05:30 [Info] logWriterStat:: 0 FlushedCount 3300000 QueuedCount 49981
2016-04-06T08:18:28.655+05:30 [Info] logWriterStat:: 0 FlushedCount 3310000 QueuedCount 49854
2016-04-06T08:18:28.759+05:30 [Info] logWriterStat:: 0 FlushedCount 3320000 QueuedCount 49997
2016-04-06T08:18:28.878+05:30 [Info] logWriterStat:: 0 FlushedCount 3330000 QueuedCount 49962
2016-04-06T08:18:28.994+05:30 [Info] logWriterStat:: 0 FlushedCount 3340000 QueuedCount 50000
2016-04-06T08:18:29.117+05:30 [Info] logWriterStat:: 0 FlushedCount 3350000 QueuedCount 49998
2016-04-06T08:18:29.242+05:30 [Info] logWriterStat:: 0 FlushedCount 3360000 QueuedCount 49972
2016-04-06T08:18:29.388+05:30 [Info] logWriterStat:: 0 FlushedCount 3370000 QueuedCount 49987
2016-04-06T08:18:29.502+05:30 [Info] logWriterStat:: 0 FlushedCount 3380000 QueuedCount 49948
2016-04-06T08:18:29.572+05:30 [Info] logWriterStat:: 0 FlushedCount 3390000 QueuedCount 49989
2016-04-06T08:18:29.644+05:30 [Info] logWriterStat:: 0 FlushedCount 3400000 QueuedCount 49979
2016-04-06T08:18:29.718+05:30 [Info] logWriterStat:: 0 FlushedCount 3410000 QueuedCount 49960
2016-04-06T08:18:29.793+05:30 [Info] logWriterStat:: 0 FlushedCount 3420000 QueuedCount 49940
2016-04-06T08:18:29.867+05:30 [Info] logWriterStat:: 0 FlushedCount 3430000 QueuedCount 49984
2016-04-06T08:18:29.943+05:30 [Info] logWriterStat:: 0 FlushedCount 3440000 QueuedCount 49903
2016-04-06T08:18:30.02+05:30 [Info] logWriterStat:: 0 FlushedCount 3450000 QueuedCount 49974
2016-04-06T08:18:30.098+05:30 [Info] logWriterStat:: 0 FlushedCount 3460000 QueuedCount 49591
2016-04-06T08:18:30.175+05:30 [Info] logWriterStat:: 0 FlushedCount 3470000 QueuedCount 49979
2016-04-06T08:18:30.251+05:30 [Info] logWriterStat:: 0 FlushedCount 3480000 QueuedCount 49934
2016-04-06T08:18:30.327+05:30 [Info] logWriterStat:: 0 FlushedCount 3490000 QueuedCount 49959
2016-04-06T08:18:30.399+05:30 [Info] logWriterStat:: 0 FlushedCount 3500000 QueuedCount 49963
2016-04-06T08:18:30.504+05:30 [Info] logWriterStat:: 0 FlushedCount 3510000 QueuedCount 49978
2016-04-06T08:18:30.601+05:30 [Info] logWriterStat:: 0 FlushedCount 3520000 QueuedCount 49956
2016-04-06T08:18:30.677+05:30 [Info] logWriterStat:: 0 FlushedCount 3530000 QueuedCount 49955
2016-04-06T08:18:30.764+05:30 [Info] logWriterStat:: 0 FlushedCount 3540000 QueuedCount 49989
2016-04-06T08:18:30.838+05:30 [Info] logWriterStat:: 0 FlushedCount 3550000 QueuedCount 49938
2016-04-06T08:18:30.909+05:30 [Info] logWriterStat:: 0 FlushedCount 3560000 QueuedCount 49941
2016-04-06T08:18:30.985+05:30 [Info] logWriterStat:: 0 FlushedCount 3570000 QueuedCount 49922
2016-04-06T08:18:31.058+05:30 [Info] logWriterStat:: 0 FlushedCount 3580000 QueuedCount 49981
2016-04-06T08:18:31.133+05:30 [Info] logWriterStat:: 0 FlushedCount 3590000 QueuedCount 49922
2016-04-06T08:18:31.204+05:30 [Info] logWriterStat:: 0 FlushedCount 3600000 QueuedCount 49983
2016-04-06T08:18:31.277+05:30 [Info] logWriterStat:: 0 FlushedCount 3610000 QueuedCount 49936
2016-04-06T08:18:31.351+05:30 [Info] logWriterStat:: 0 FlushedCount 3620000 QueuedCount 49991
2016-04-06T08:18:31.425+05:30 [Info] logWriterStat:: 0 FlushedCount 3630000 QueuedCount 49976
2016-04-06T08:18:31.51+05:30 [Info] logWriterStat:: 0 FlushedCount 3640000 QueuedCount 49988
2016-04-06T08:18:31.581+05:30 [Info] logWriterStat:: 0 FlushedCount 3650000 QueuedCount 49964
2016-04-06T08:18:31.65+05:30 [Info] logWriterStat:: 0 FlushedCount 3660000 QueuedCount 48889
2016-04-06T08:18:31.719+05:30 [Info] logWriterStat:: 0 FlushedCount 3670000 QueuedCount 49954
2016-04-06T08:18:31.793+05:30 [Info] logWriterStat:: 0 FlushedCount 3680000 QueuedCount 49978
2016-04-06T08:18:31.863+05:30 [Info] logWriterStat:: 0 FlushedCount 3690000 QueuedCount 49966
2016-04-06T08:18:31.937+05:30 [Info] logWriterStat:: 0 FlushedCount 3700000 QueuedCount 49982
2016-04-06T08:18:32.02+05:30 [Info] logWriterStat:: 0 FlushedCount 3710000 QueuedCount 49957
2016-04-06T08:18:32.093+05:30 [Info] logWriterStat:: 0 FlushedCount 3720000 QueuedCount 49944
2016-04-06T08:18:32.165+05:30 [Info] logWriterStat:: 0 FlushedCount 3730000 QueuedCount 49962
2016-04-06T08:18:32.232+05:30 [Info] logWriterStat:: 0 FlushedCount 3740000 QueuedCount 49942
2016-04-06T08:18:32.302+05:30 [Info] logWriterStat:: 0 FlushedCount 3750000 QueuedCount 49530
2016-04-06T08:18:32.368+05:30 [Info] logWriterStat:: 0 FlushedCount 3760000 QueuedCount 49949
2016-04-06T08:18:32.454+05:30 [Info] logWriterStat:: 0 FlushedCount 3770000 QueuedCount 49932
2016-04-06T08:18:32.527+05:30 [Info] logWriterStat:: 0 FlushedCount 3780000 QueuedCount 49976
2016-04-06T08:18:32.601+05:30 [Info] logWriterStat:: 0 FlushedCount 3790000 QueuedCount 49893
2016-04-06T08:18:32.671+05:30 [Info] logWriterStat:: 0 FlushedCount 3800000 QueuedCount 49931
2016-04-06T08:18:32.74+05:30 [Info] logWriterStat:: 0 FlushedCount 3810000 QueuedCount 49969
2016-04-06T08:18:32.809+05:30 [Info] logWriterStat:: 0 FlushedCount 3820000 QueuedCount 49941
2016-04-06T08:18:32.88+05:30 [Info] logWriterStat:: 0 FlushedCount 3830000 QueuedCount 49954
2016-04-06T08:18:32.95+05:30 [Info] logWriterStat:: 0 FlushedCount 3840000 QueuedCount 49914
2016-04-06T08:18:33.025+05:30 [Info] logWriterStat:: 0 FlushedCount 3850000 QueuedCount 49896
2016-04-06T08:18:33.089+05:30 [Info] logWriterStat:: 0 FlushedCount 3860000 QueuedCount 49987
2016-04-06T08:18:33.159+05:30 [Info] logWriterStat:: 0 FlushedCount 3870000 QueuedCount 49969
2016-04-06T08:18:33.228+05:30 [Info] logWriterStat:: 0 FlushedCount 3880000 QueuedCount 49944
2016-04-06T08:18:33.293+05:30 [Info] logWriterStat:: 0 FlushedCount 3890000 QueuedCount 49904
2016-04-06T08:18:33.363+05:30 [Info] logWriterStat:: 0 FlushedCount 3900000 QueuedCount 49961
2016-04-06T08:18:33.429+05:30 [Info] logWriterStat:: 0 FlushedCount 3910000 QueuedCount 49861
2016-04-06T08:18:33.495+05:30 [Info] logWriterStat:: 0 FlushedCount 3920000 QueuedCount 45572
2016-04-06T08:18:33.56+05:30 [Info] logWriterStat:: 0 FlushedCount 3930000 QueuedCount 40764
2016-04-06T08:18:33.626+05:30 [Info] logWriterStat:: 0 FlushedCount 3940000 QueuedCount 49951
2016-04-06T08:18:33.704+05:30 [Info] logWriterStat:: 0 FlushedCount 3950000 QueuedCount 49490
2016-04-06T08:18:33.779+05:30 [Info] logWriterStat:: 0 FlushedCount 3960000 QueuedCount 49969
2016-04-06T08:18:33.847+05:30 [Info] logWriterStat:: 0 FlushedCount 3970000 QueuedCount 49951
2016-04-06T08:18:33.918+05:30 [Info] logWriterStat:: 0 FlushedCount 3980000 QueuedCount 49983
2016-04-06T08:18:33.99+05:30 [Info] logWriterStat:: 0 FlushedCount 3990000 QueuedCount 49993
2016-04-06T08:18:34.073+05:30 [Info] logWriterStat:: 0 FlushedCount 4000000 QueuedCount 49923
2016-04-06T08:18:34.143+05:30 [Info] logWriterStat:: 0 FlushedCount 4010000 QueuedCount 49942
2016-04-06T08:18:34.212+05:30 [Info] logWriterStat:: 0 FlushedCount 4020000 QueuedCount 49942
2016-04-06T08:18:34.281+05:30 [Info] logWriterStat:: 0 FlushedCount 4030000 QueuedCount 49418
2016-04-06T08:18:34.346+05:30 [Info] logWriterStat:: 0 FlushedCount 4040000 QueuedCount 49984
2016-04-06T08:18:34.413+05:30 [Info] logWriterStat:: 0 FlushedCount 4050000 QueuedCount 49966
2016-04-06T08:18:34.493+05:30 [Info] logWriterStat:: 0 FlushedCount 4060000 QueuedCount 49954
2016-04-06T08:18:34.561+05:30 [Info] logWriterStat:: 0 FlushedCount 4070000 QueuedCount 49964
2016-04-06T08:18:34.627+05:30 [Info] logWriterStat:: 0 FlushedCount 4080000 QueuedCount 49977
2016-04-06T08:18:34.694+05:30 [Info] logWriterStat:: 0 FlushedCount 4090000 QueuedCount 49902
2016-04-06T08:18:34.765+05:30 [Info] logWriterStat:: 0 FlushedCount 4100000 QueuedCount 49871
2016-04-06T08:18:34.835+05:30 [Info] logWriterStat:: 0 FlushedCount 4110000 QueuedCount 49961
2016-04-06T08:18:34.908+05:30 [Info] logWriterStat:: 0 FlushedCount 4120000 QueuedCount 49979
2016-04-06T08:18:34.976+05:30 [Info] logWriterStat:: 0 FlushedCount 4130000 QueuedCount 49969
2016-04-06T08:18:35.046+05:30 [Info] logWriterStat:: 0 FlushedCount 4140000 QueuedCount 49984
2016-04-06T08:18:35.115+05:30 [Info] logWriterStat:: 0 FlushedCount 4150000 QueuedCount 49958
2016-04-06T08:18:35.182+05:30 [Info] logWriterStat:: 0 FlushedCount 4160000 QueuedCount 49947
2016-04-06T08:18:35.25+05:30 [Info] logWriterStat:: 0 FlushedCount 4170000 QueuedCount 49930
2016-04-06T08:18:35.312+05:30 [Info] logWriterStat:: 0 FlushedCount 4180000 QueuedCount 49978
2016-04-06T08:18:35.38+05:30 [Info] logWriterStat:: 0 FlushedCount 4190000 QueuedCount 49952
2016-04-06T08:18:35.458+05:30 [Info] logWriterStat:: 0 FlushedCount 4200000 QueuedCount 49931
2016-04-06T08:18:35.524+05:30 [Info] logWriterStat:: 0 FlushedCount 4210000 QueuedCount 49917
2016-04-06T08:18:35.595+05:30 [Info] logWriterStat:: 0 FlushedCount 4220000 QueuedCount 49979
2016-04-06T08:18:35.663+05:30 [Info] logWriterStat:: 0 FlushedCount 4230000 QueuedCount 49874
2016-04-06T08:18:35.747+05:30 [Info] logWriterStat:: 0 FlushedCount 4240000 QueuedCount 49978
2016-04-06T08:18:35.817+05:30 [Info] logWriterStat:: 0 FlushedCount 4250000 QueuedCount 49946
2016-04-06T08:18:35.886+05:30 [Info] logWriterStat:: 0 FlushedCount 4260000 QueuedCount 49946
2016-04-06T08:18:35.957+05:30 [Info] logWriterStat:: 0 FlushedCount 4270000 QueuedCount 49964
2016-04-06T08:18:36.025+05:30 [Info] logWriterStat:: 0 FlushedCount 4280000 QueuedCount 49986
2016-04-06T08:18:36.093+05:30 [Info] logWriterStat:: 0 FlushedCount 4290000 QueuedCount 50000
2016-04-06T08:18:36.163+05:30 [Info] logWriterStat:: 0 FlushedCount 4300000 QueuedCount 49946
2016-04-06T08:18:36.232+05:30 [Info] logWriterStat:: 0 FlushedCount 4310000 QueuedCount 49956
2016-04-06T08:18:36.302+05:30 [Info] logWriterStat:: 0 FlushedCount 4320000 QueuedCount 49956
2016-04-06T08:18:36.37+05:30 [Info] logWriterStat:: 0 FlushedCount 4330000 QueuedCount 49963
2016-04-06T08:18:36.441+05:30 [Info] logWriterStat:: 0 FlushedCount 4340000 QueuedCount 49963
2016-04-06T08:18:36.518+05:30 [Info] logWriterStat:: 0 FlushedCount 4350000 QueuedCount 49988
2016-04-06T08:18:36.588+05:30 [Info] logWriterStat:: 0 FlushedCount 4360000 QueuedCount 49981
2016-04-06T08:18:36.652+05:30 [Info] logWriterStat:: 0 FlushedCount 4370000 QueuedCount 49592
2016-04-06T08:18:36.723+05:30 [Info] logWriterStat:: 0 FlushedCount 4380000 QueuedCount 49944
2016-04-06T08:18:36.794+05:30 [Info] logWriterStat:: 0 FlushedCount 4390000 QueuedCount 49897
2016-04-06T08:18:36.861+05:30 [Info] logWriterStat:: 0 FlushedCount 4400000 QueuedCount 49971
2016-04-06T08:18:36.929+05:30 [Info] logWriterStat:: 0 FlushedCount 4410000 QueuedCount 49841
2016-04-06T08:18:36.997+05:30 [Info] logWriterStat:: 0 FlushedCount 4420000 QueuedCount 49022
2016-04-06T08:18:37.074+05:30 [Info] logWriterStat:: 0 FlushedCount 4430000 QueuedCount 49964
2016-04-06T08:18:37.144+05:30 [Info] logWriterStat:: 0 FlushedCount 4440000 QueuedCount 49960
2016-04-06T08:18:37.21+05:30 [Info] logWriterStat:: 0 FlushedCount 4450000 QueuedCount 49896
2016-04-06T08:18:37.279+05:30 [Info] logWriterStat:: 0 FlushedCount 4460000 QueuedCount 49913
2016-04-06T08:18:37.347+05:30 [Info] logWriterStat:: 0 FlushedCount 4470000 QueuedCount 49954
2016-04-06T08:18:37.417+05:30 [Info] logWriterStat:: 0 FlushedCount 4480000 QueuedCount 49925
2016-04-06T08:18:37.491+05:30 [Info] logWriterStat:: 0 FlushedCount 4490000 QueuedCount 49903
2016-04-06T08:18:37.561+05:30 [Info] logWriterStat:: 0 FlushedCount 4500000 QueuedCount 49915
2016-04-06T08:18:37.632+05:30 [Info] logWriterStat:: 0 FlushedCount 4510000 QueuedCount 49965
2016-04-06T08:18:37.699+05:30 [Info] logWriterStat:: 0 FlushedCount 4520000 QueuedCount 49952
2016-04-06T08:18:37.768+05:30 [Info] logWriterStat:: 0 FlushedCount 4530000 QueuedCount 49912
2016-04-06T08:18:37.836+05:30 [Info] logWriterStat:: 0 FlushedCount 4540000 QueuedCount 49927
2016-04-06T08:18:37.905+05:30 [Info] logWriterStat:: 0 FlushedCount 4550000 QueuedCount 49950
2016-04-06T08:18:37.974+05:30 [Info] logWriterStat:: 0 FlushedCount 4560000 QueuedCount 49972
2016-04-06T08:18:38.046+05:30 [Info] logWriterStat:: 0 FlushedCount 4570000 QueuedCount 49950
2016-04-06T08:18:38.114+05:30 [Info] logWriterStat:: 0 FlushedCount 4580000 QueuedCount 49961
2016-04-06T08:18:38.183+05:30 [Info] logWriterStat:: 0 FlushedCount 4590000 QueuedCount 49946
2016-04-06T08:18:38.252+05:30 [Info] logWriterStat:: 0 FlushedCount 4600000 QueuedCount 49903
2016-04-06T08:18:38.32+05:30 [Info] logWriterStat:: 0 FlushedCount 4610000 QueuedCount 49925
2016-04-06T08:18:38.387+05:30 [Info] logWriterStat:: 0 FlushedCount 4620000 QueuedCount 49835
2016-04-06T08:18:38.465+05:30 [Info] logWriterStat:: 0 FlushedCount 4630000 QueuedCount 49918
2016-04-06T08:18:38.535+05:30 [Info] logWriterStat:: 0 FlushedCount 4640000 QueuedCount 49887
2016-04-06T08:18:38.604+05:30 [Info] logWriterStat:: 0 FlushedCount 4650000 QueuedCount 49947
2016-04-06T08:18:38.672+05:30 [Info] logWriterStat:: 0 FlushedCount 4660000 QueuedCount 49986
2016-04-06T08:18:38.742+05:30 [Info] logWriterStat:: 0 FlushedCount 4670000 QueuedCount 49946
2016-04-06T08:18:38.816+05:30 [Info] logWriterStat:: 0 FlushedCount 4680000 QueuedCount 49876
2016-04-06T08:18:38.892+05:30 [Info] logWriterStat:: 0 FlushedCount 4690000 QueuedCount 49892
2016-04-06T08:18:38.967+05:30 [Info] logWriterStat:: 0 FlushedCount 4700000 QueuedCount 49998
2016-04-06T08:18:39.053+05:30 [Info] logWriterStat:: 0 FlushedCount 4710000 QueuedCount 49944
2016-04-06T08:18:39.127+05:30 [Info] logWriterStat:: 0 FlushedCount 4720000 QueuedCount 49961
2016-04-06T08:18:39.197+05:30 [Info] logWriterStat:: 0 FlushedCount 4730000 QueuedCount 49987
2016-04-06T08:18:39.263+05:30 [Info] logWriterStat:: 0 FlushedCount 4740000 QueuedCount 49999
2016-04-06T08:18:39.33+05:30 [Info] logWriterStat:: 0 FlushedCount 4750000 QueuedCount 49936
2016-04-06T08:18:39.397+05:30 [Info] logWriterStat:: 0 FlushedCount 4760000 QueuedCount 49945
2016-04-06T08:18:39.476+05:30 [Info] logWriterStat:: 0 FlushedCount 4770000 QueuedCount 49951
2016-04-06T08:18:39.542+05:30 [Info] logWriterStat:: 0 FlushedCount 4780000 QueuedCount 49963
2016-04-06T08:18:39.611+05:30 [Info] logWriterStat:: 0 FlushedCount 4790000 QueuedCount 49974
2016-04-06T08:18:39.681+05:30 [Info] logWriterStat:: 0 FlushedCount 4800000 QueuedCount 49995
2016-04-06T08:18:39.757+05:30 [Info] logWriterStat:: 0 FlushedCount 4810000 QueuedCount 49910
2016-04-06T08:18:39.821+05:30 [Info] logWriterStat:: 0 FlushedCount 4820000 QueuedCount 48086
2016-04-06T08:18:39.89+05:30 [Info] logWriterStat:: 0 FlushedCount 4830000 QueuedCount 49969
2016-04-06T08:18:39.959+05:30 [Info] logWriterStat:: 0 FlushedCount 4840000 QueuedCount 49955
2016-04-06T08:18:40.028+05:30 [Info] logWriterStat:: 0 FlushedCount 4850000 QueuedCount 49897
2016-04-06T08:18:40.095+05:30 [Info] logWriterStat:: 0 FlushedCount 4860000 QueuedCount 49961
2016-04-06T08:18:40.165+05:30 [Info] logWriterStat:: 0 FlushedCount 4870000 QueuedCount 49981
2016-04-06T08:18:40.236+05:30 [Info] logWriterStat:: 0 FlushedCount 4880000 QueuedCount 49962
2016-04-06T08:18:40.304+05:30 [Info] logWriterStat:: 0 FlushedCount 4890000 QueuedCount 49967
2016-04-06T08:18:40.373+05:30 [Info] logWriterStat:: 0 FlushedCount 4900000 QueuedCount 49940
2016-04-06T08:18:40.455+05:30 [Info] logWriterStat:: 0 FlushedCount 4910000 QueuedCount 49998
2016-04-06T08:18:40.569+05:30 [Info] logWriterStat:: 0 FlushedCount 4920000 QueuedCount 49962
2016-04-06T08:18:40.637+05:30 [Info] logWriterStat:: 0 FlushedCount 4930000 QueuedCount 49970
2016-04-06T08:18:40.707+05:30 [Info] logWriterStat:: 0 FlushedCount 4940000 QueuedCount 49990
2016-04-06T08:18:40.787+05:30 [Info] logWriterStat:: 0 FlushedCount 4950000 QueuedCount 49952
2016-04-06T08:18:40.856+05:30 [Info] logWriterStat:: 0 FlushedCount 4960000 QueuedCount 49877
2016-04-06T08:18:40.926+05:30 [Info] logWriterStat:: 0 FlushedCount 4970000 QueuedCount 49954
2016-04-06T08:18:40.996+05:30 [Info] logWriterStat:: 0 FlushedCount 4980000 QueuedCount 49960
2016-04-06T08:18:41.065+05:30 [Info] logWriterStat:: 0 FlushedCount 4990000 QueuedCount 49987
2016-04-06T08:18:41.131+05:30 [Info] logWriterStat:: 0 FlushedCount 5000000 QueuedCount 49981
2016-04-06T08:18:41.2+05:30 [Info] logWriterStat:: 0 FlushedCount 5010000 QueuedCount 49949
2016-04-06T08:18:41.271+05:30 [Info] logWriterStat:: 0 FlushedCount 5020000 QueuedCount 49968
2016-04-06T08:18:41.339+05:30 [Info] logWriterStat:: 0 FlushedCount 5030000 QueuedCount 49925
2016-04-06T08:18:41.41+05:30 [Info] logWriterStat:: 0 FlushedCount 5040000 QueuedCount 49989
2016-04-06T08:18:41.491+05:30 [Info] logWriterStat:: 0 FlushedCount 5050000 QueuedCount 49957
2016-04-06T08:18:41.561+05:30 [Info] logWriterStat:: 0 FlushedCount 5060000 QueuedCount 49924
2016-04-06T08:18:41.629+05:30 [Info] logWriterStat:: 0 FlushedCount 5070000 QueuedCount 49871
2016-04-06T08:18:41.698+05:30 [Info] logWriterStat:: 0 FlushedCount 5080000 QueuedCount 49972
2016-04-06T08:18:41.769+05:30 [Info] logWriterStat:: 0 FlushedCount 5090000 QueuedCount 49928
2016-04-06T08:18:41.84+05:30 [Info] logWriterStat:: 0 FlushedCount 5100000 QueuedCount 49946
2016-04-06T08:18:41.909+05:30 [Info] logWriterStat:: 0 FlushedCount 5110000 QueuedCount 49920
2016-04-06T08:18:41.98+05:30 [Info] logWriterStat:: 0 FlushedCount 5120000 QueuedCount 49998
2016-04-06T08:18:42.061+05:30 [Info] logWriterStat:: 0 FlushedCount 5130000 QueuedCount 49933
2016-04-06T08:18:42.128+05:30 [Info] logWriterStat:: 0 FlushedCount 5140000 QueuedCount 49958
2016-04-06T08:18:42.198+05:30 [Info] logWriterStat:: 0 FlushedCount 5150000 QueuedCount 49971
2016-04-06T08:18:42.269+05:30 [Info] logWriterStat:: 0 FlushedCount 5160000 QueuedCount 49989
2016-04-06T08:18:42.339+05:30 [Info] logWriterStat:: 0 FlushedCount 5170000 QueuedCount 49961
2016-04-06T08:18:42.407+05:30 [Info] logWriterStat:: 0 FlushedCount 5180000 QueuedCount 49884
2016-04-06T08:18:42.485+05:30 [Info] logWriterStat:: 0 FlushedCount 5190000 QueuedCount 49930
2016-04-06T08:18:42.552+05:30 [Info] logWriterStat:: 0 FlushedCount 5200000 QueuedCount 49948
2016-04-06T08:18:42.625+05:30 [Info] logWriterStat:: 0 FlushedCount 5210000 QueuedCount 49969
2016-04-06T08:18:42.697+05:30 [Info] logWriterStat:: 0 FlushedCount 5220000 QueuedCount 49988
2016-04-06T08:18:42.771+05:30 [Info] logWriterStat:: 0 FlushedCount 5230000 QueuedCount 49922
2016-04-06T08:18:42.839+05:30 [Info] logWriterStat:: 0 FlushedCount 5240000 QueuedCount 49955
2016-04-06T08:18:42.908+05:30 [Info] logWriterStat:: 0 FlushedCount 5250000 QueuedCount 49932
2016-04-06T08:18:42.98+05:30 [Info] logWriterStat:: 0 FlushedCount 5260000 QueuedCount 49965
2016-04-06T08:18:43.055+05:30 [Info] logWriterStat:: 0 FlushedCount 5270000 QueuedCount 49798
2016-04-06T08:18:43.126+05:30 [Info] logWriterStat:: 0 FlushedCount 5280000 QueuedCount 49965
2016-04-06T08:18:43.193+05:30 [Info] logWriterStat:: 0 FlushedCount 5290000 QueuedCount 49958
2016-04-06T08:18:43.262+05:30 [Info] logWriterStat:: 0 FlushedCount 5300000 QueuedCount 49971
2016-04-06T08:18:43.331+05:30 [Info] logWriterStat:: 0 FlushedCount 5310000 QueuedCount 49916
2016-04-06T08:18:43.4+05:30 [Info] logWriterStat:: 0 FlushedCount 5320000 QueuedCount 49995
2016-04-06T08:18:43.481+05:30 [Info] logWriterStat:: 0 FlushedCount 5330000 QueuedCount 49996
2016-04-06T08:18:43.549+05:30 [Info] logWriterStat:: 0 FlushedCount 5340000 QueuedCount 49950
2016-04-06T08:18:43.613+05:30 [Info] logWriterStat:: 0 FlushedCount 5350000 QueuedCount 49914
2016-04-06T08:18:43.681+05:30 [Info] logWriterStat:: 0 FlushedCount 5360000 QueuedCount 49948
2016-04-06T08:18:43.756+05:30 [Info] logWriterStat:: 0 FlushedCount 5370000 QueuedCount 49466
2016-04-06T08:18:43.827+05:30 [Info] logWriterStat:: 0 FlushedCount 5380000 QueuedCount 49906
2016-04-06T08:18:43.893+05:30 [Info] logWriterStat:: 0 FlushedCount 5390000 QueuedCount 48413
2016-04-06T08:18:43.961+05:30 [Info] logWriterStat:: 0 FlushedCount 5400000 QueuedCount 49991
2016-04-06T08:18:44.052+05:30 [Info] logWriterStat:: 0 FlushedCount 5410000 QueuedCount 49978
2016-04-06T08:18:44.122+05:30 [Info] logWriterStat:: 0 FlushedCount 5420000 QueuedCount 49979
2016-04-06T08:18:44.184+05:30 [Info] logWriterStat:: 0 FlushedCount 5430000 QueuedCount 49948
2016-04-06T08:18:44.252+05:30 [Info] logWriterStat:: 0 FlushedCount 5440000 QueuedCount 49899
2016-04-06T08:18:44.321+05:30 [Info] logWriterStat:: 0 FlushedCount 5450000 QueuedCount 49914
2016-04-06T08:18:44.39+05:30 [Info] logWriterStat:: 0 FlushedCount 5460000 QueuedCount 49931
2016-04-06T08:18:44.471+05:30 [Info] logWriterStat:: 0 FlushedCount 5470000 QueuedCount 49954
2016-04-06T08:18:44.542+05:30 [Info] logWriterStat:: 0 FlushedCount 5480000 QueuedCount 49963
2016-04-06T08:18:44.612+05:30 [Info] logWriterStat:: 0 FlushedCount 5490000 QueuedCount 49955
2016-04-06T08:18:44.679+05:30 [Info] logWriterStat:: 0 FlushedCount 5500000 QueuedCount 49891
2016-04-06T08:18:44.746+05:30 [Info] logWriterStat:: 0 FlushedCount 5510000 QueuedCount 49983
2016-04-06T08:18:44.817+05:30 [Info] logWriterStat:: 0 FlushedCount 5520000 QueuedCount 49922
2016-04-06T08:18:44.881+05:30 [Info] logWriterStat:: 0 FlushedCount 5530000 QueuedCount 49914
2016-04-06T08:18:44.953+05:30 [Info] logWriterStat:: 0 FlushedCount 5540000 QueuedCount 49977
2016-04-06T08:18:45.022+05:30 [Info] logWriterStat:: 0 FlushedCount 5550000 QueuedCount 49967
2016-04-06T08:18:45.091+05:30 [Info] logWriterStat:: 0 FlushedCount 5560000 QueuedCount 49929
2016-04-06T08:18:45.16+05:30 [Info] logWriterStat:: 0 FlushedCount 5570000 QueuedCount 49968
2016-04-06T08:18:45.23+05:30 [Info] logWriterStat:: 0 FlushedCount 5580000 QueuedCount 49908
2016-04-06T08:18:45.301+05:30 [Info] logWriterStat:: 0 FlushedCount 5590000 QueuedCount 49984
2016-04-06T08:18:45.371+05:30 [Info] logWriterStat:: 0 FlushedCount 5600000 QueuedCount 49958
2016-04-06T08:18:45.449+05:30 [Info] logWriterStat:: 0 FlushedCount 5610000 QueuedCount 49240
2016-04-06T08:18:45.521+05:30 [Info] logWriterStat:: 0 FlushedCount 5620000 QueuedCount 49953
2016-04-06T08:18:45.591+05:30 [Info] logWriterStat:: 0 FlushedCount 5630000 QueuedCount 49955
2016-04-06T08:18:45.66+05:30 [Info] logWriterStat:: 0 FlushedCount 5640000 QueuedCount 49977
2016-04-06T08:18:45.74+05:30 [Info] logWriterStat:: 0 FlushedCount 5650000 QueuedCount 49984
2016-04-06T08:18:45.815+05:30 [Info] logWriterStat:: 0 FlushedCount 5660000 QueuedCount 49975
2016-04-06T08:18:45.887+05:30 [Info] logWriterStat:: 0 FlushedCount 5670000 QueuedCount 49955
2016-04-06T08:18:45.955+05:30 [Info] logWriterStat:: 0 FlushedCount 5680000 QueuedCount 49971
2016-04-06T08:18:46.026+05:30 [Info] logWriterStat:: 0 FlushedCount 5690000 QueuedCount 49969
2016-04-06T08:18:46.096+05:30 [Info] logWriterStat:: 0 FlushedCount 5700000 QueuedCount 49971
2016-04-06T08:18:46.17+05:30 [Info] logWriterStat:: 0 FlushedCount 5710000 QueuedCount 49984
2016-04-06T08:18:46.238+05:30 [Info] logWriterStat:: 0 FlushedCount 5720000 QueuedCount 49883
2016-04-06T08:18:46.307+05:30 [Info] logWriterStat:: 0 FlushedCount 5730000 QueuedCount 49891
2016-04-06T08:18:46.376+05:30 [Info] logWriterStat:: 0 FlushedCount 5740000 QueuedCount 49922
2016-04-06T08:18:46.454+05:30 [Info] logWriterStat:: 0 FlushedCount 5750000 QueuedCount 49973
2016-04-06T08:18:46.524+05:30 [Info] logWriterStat:: 0 FlushedCount 5760000 QueuedCount 49995
2016-04-06T08:18:46.592+05:30 [Info] logWriterStat:: 0 FlushedCount 5770000 QueuedCount 49989
2016-04-06T08:18:46.661+05:30 [Info] logWriterStat:: 0 FlushedCount 5780000 QueuedCount 49998
2016-04-06T08:18:46.729+05:30 [Info] logWriterStat:: 0 FlushedCount 5790000 QueuedCount 49848
2016-04-06T08:18:46.805+05:30 [Info] logWriterStat:: 0 FlushedCount 5800000 QueuedCount 49948
2016-04-06T08:18:46.875+05:30 [Info] logWriterStat:: 0 FlushedCount 5810000 QueuedCount 49968
2016-04-06T08:18:46.948+05:30 [Info] logWriterStat:: 0 FlushedCount 5820000 QueuedCount 49972
2016-04-06T08:18:47.017+05:30 [Info] logWriterStat:: 0 FlushedCount 5830000 QueuedCount 49958
2016-04-06T08:18:47.092+05:30 [Info] logWriterStat:: 0 FlushedCount 5840000 QueuedCount 49932
2016-04-06T08:18:47.164+05:30 [Info] logWriterStat:: 0 FlushedCount 5850000 QueuedCount 49937
2016-04-06T08:18:47.23+05:30 [Info] logWriterStat:: 0 FlushedCount 5860000 QueuedCount 49944
2016-04-06T08:18:47.298+05:30 [Info] logWriterStat:: 0 FlushedCount 5870000 QueuedCount 49981
2016-04-06T08:18:47.365+05:30 [Info] logWriterStat:: 0 FlushedCount 5880000 QueuedCount 49967
2016-04-06T08:18:47.432+05:30 [Info] logWriterStat:: 0 FlushedCount 5890000 QueuedCount 49926
2016-04-06T08:18:47.509+05:30 [Info] logWriterStat:: 0 FlushedCount 5900000 QueuedCount 49837
2016-04-06T08:18:47.578+05:30 [Info] logWriterStat:: 0 FlushedCount 5910000 QueuedCount 49446
2016-04-06T08:18:47.65+05:30 [Info] logWriterStat:: 0 FlushedCount 5920000 QueuedCount 49955
2016-04-06T08:18:47.715+05:30 [Info] logWriterStat:: 0 FlushedCount 5930000 QueuedCount 49911
2016-04-06T08:18:47.783+05:30 [Info] logWriterStat:: 0 FlushedCount 5940000 QueuedCount 49947
2016-04-06T08:18:47.851+05:30 [Info] logWriterStat:: 0 FlushedCount 5950000 QueuedCount 49950
2016-04-06T08:18:47.921+05:30 [Info] logWriterStat:: 0 FlushedCount 5960000 QueuedCount 49973
2016-04-06T08:18:47.99+05:30 [Info] logWriterStat:: 0 FlushedCount 5970000 QueuedCount 49918
2016-04-06T08:18:48.057+05:30 [Info] logWriterStat:: 0 FlushedCount 5980000 QueuedCount 47597
2016-04-06T08:18:48.125+05:30 [Info] logWriterStat:: 0 FlushedCount 5990000 QueuedCount 49902
2016-04-06T08:18:48.195+05:30 [Info] logWriterStat:: 0 FlushedCount 6000000 QueuedCount 49973
2016-04-06T08:18:48.266+05:30 [Info] logWriterStat:: 0 FlushedCount 6010000 QueuedCount 49904
2016-04-06T08:18:48.333+05:30 [Info] logWriterStat:: 0 FlushedCount 6020000 QueuedCount 49922
2016-04-06T08:18:48.403+05:30 [Info] logWriterStat:: 0 FlushedCount 6030000 QueuedCount 49936
2016-04-06T08:18:48.483+05:30 [Info] logWriterStat:: 0 FlushedCount 6040000 QueuedCount 49892
2016-04-06T08:18:48.555+05:30 [Info] logWriterStat:: 0 FlushedCount 6050000 QueuedCount 49948
2016-04-06T08:18:48.627+05:30 [Info] logWriterStat:: 0 FlushedCount 6060000 QueuedCount 49985
2016-04-06T08:18:48.702+05:30 [Info] logWriterStat:: 0 FlushedCount 6070000 QueuedCount 49979
2016-04-06T08:18:48.774+05:30 [Info] logWriterStat:: 0 FlushedCount 6080000 QueuedCount 49902
2016-04-06T08:18:48.841+05:30 [Info] logWriterStat:: 0 FlushedCount 6090000 QueuedCount 49947
2016-04-06T08:18:48.913+05:30 [Info] logWriterStat:: 0 FlushedCount 6100000 QueuedCount 49946
2016-04-06T08:18:49.104+05:30 [Info] logWriterStat:: 0 FlushedCount 6110000 QueuedCount 49965
2016-04-06T08:18:49.294+05:30 [Info] logWriterStat:: 0 FlushedCount 6120000 QueuedCount 49970
2016-04-06T08:18:49.513+05:30 [Info] logWriterStat:: 0 FlushedCount 6130000 QueuedCount 48959
2016-04-06T08:18:49.754+05:30 [Info] logWriterStat:: 0 FlushedCount 6140000 QueuedCount 48930
2016-04-06T08:18:49.836+05:30 [Info] logWriterStat:: 0 FlushedCount 6150000 QueuedCount 44478
2016-04-06T08:18:49.901+05:30 [Info] logWriterStat:: 0 FlushedCount 6160000 QueuedCount 39974
2016-04-06T08:18:49.971+05:30 [Info] logWriterStat:: 0 FlushedCount 6170000 QueuedCount 37531
2016-04-06T08:18:50.037+05:30 [Info] logWriterStat:: 0 FlushedCount 6180000 QueuedCount 37168
2016-04-06T08:18:50.106+05:30 [Info] logWriterStat:: 0 FlushedCount 6190000 QueuedCount 40893
2016-04-06T08:18:50.179+05:30 [Info] logWriterStat:: 0 FlushedCount 6200000 QueuedCount 40838
2016-04-06T08:18:50.245+05:30 [Info] logWriterStat:: 0 FlushedCount 6210000 QueuedCount 37006
2016-04-06T08:18:50.315+05:30 [Info] logWriterStat:: 0 FlushedCount 6220000 QueuedCount 37204
2016-04-06T08:18:50.385+05:30 [Info] logWriterStat:: 0 FlushedCount 6230000 QueuedCount 38206
2016-04-06T08:18:50.481+05:30 [Info] logWriterStat:: 0 FlushedCount 6240000 QueuedCount 34683
2016-04-06T08:18:50.628+05:30 [Info] logWriterStat:: 0 FlushedCount 6250000 QueuedCount 49990
2016-04-06T08:18:50.774+05:30 [Info] logWriterStat:: 0 FlushedCount 6260000 QueuedCount 49975
2016-04-06T08:18:50.89+05:30 [Info] logWriterStat:: 0 FlushedCount 6270000 QueuedCount 49324
2016-04-06T08:18:50.994+05:30 [Info] logWriterStat:: 0 FlushedCount 6280000 QueuedCount 49899
2016-04-06T08:18:51.117+05:30 [Info] logWriterStat:: 0 FlushedCount 6290000 QueuedCount 49987
2016-04-06T08:18:51.251+05:30 [Info] logWriterStat:: 0 FlushedCount 6300000 QueuedCount 49994
2016-04-06T08:18:51.364+05:30 [Info] logWriterStat:: 0 FlushedCount 6310000 QueuedCount 49901
2016-04-06T08:18:51.482+05:30 [Info] logWriterStat:: 0 FlushedCount 6320000 QueuedCount 49932
2016-04-06T08:18:51.597+05:30 [Info] logWriterStat:: 0 FlushedCount 6330000 QueuedCount 49942
2016-04-06T08:18:51.719+05:30 [Info] logWriterStat:: 0 FlushedCount 6340000 QueuedCount 49891
2016-04-06T08:18:51.836+05:30 [Info] logWriterStat:: 0 FlushedCount 6350000 QueuedCount 49825
2016-04-06T08:18:51.962+05:30 [Info] logWriterStat:: 0 FlushedCount 6360000 QueuedCount 49999
2016-04-06T08:18:52.055+05:30 [Info] logWriterStat:: 0 FlushedCount 6370000 QueuedCount 49905
2016-04-06T08:18:52.14+05:30 [Info] logWriterStat:: 0 FlushedCount 6380000 QueuedCount 49962
2016-04-06T08:18:52.205+05:30 [Info] logWriterStat:: 0 FlushedCount 6390000 QueuedCount 49998
2016-04-06T08:18:52.277+05:30 [Info] logWriterStat:: 0 FlushedCount 6400000 QueuedCount 49954
2016-04-06T08:18:52.346+05:30 [Info] logWriterStat:: 0 FlushedCount 6410000 QueuedCount 49858
2016-04-06T08:18:52.411+05:30 [Info] logWriterStat:: 0 FlushedCount 6420000 QueuedCount 49986
2016-04-06T08:18:52.491+05:30 [Info] logWriterStat:: 0 FlushedCount 6430000 QueuedCount 49976
2016-04-06T08:18:52.563+05:30 [Info] logWriterStat:: 0 FlushedCount 6440000 QueuedCount 49984
2016-04-06T08:18:52.632+05:30 [Info] logWriterStat:: 0 FlushedCount 6450000 QueuedCount 49941
2016-04-06T08:18:52.7+05:30 [Info] logWriterStat:: 0 FlushedCount 6460000 QueuedCount 49928
2016-04-06T08:18:52.77+05:30 [Info] logWriterStat:: 0 FlushedCount 6470000 QueuedCount 49971
2016-04-06T08:18:52.84+05:30 [Info] logWriterStat:: 0 FlushedCount 6480000 QueuedCount 49955
2016-04-06T08:18:52.904+05:30 [Info] logWriterStat:: 0 FlushedCount 6490000 QueuedCount 49919
2016-04-06T08:18:52.975+05:30 [Info] logWriterStat:: 0 FlushedCount 6500000 QueuedCount 49987
2016-04-06T08:18:53.05+05:30 [Info] logWriterStat:: 0 FlushedCount 6510000 QueuedCount 49956
2016-04-06T08:18:53.12+05:30 [Info] logWriterStat:: 0 FlushedCount 6520000 QueuedCount 49996
2016-04-06T08:18:53.191+05:30 [Info] logWriterStat:: 0 FlushedCount 6530000 QueuedCount 49925
2016-04-06T08:18:53.26+05:30 [Info] logWriterStat:: 0 FlushedCount 6540000 QueuedCount 49977
2016-04-06T08:18:53.334+05:30 [Info] logWriterStat:: 0 FlushedCount 6550000 QueuedCount 49956
2016-04-06T08:18:53.407+05:30 [Info] logWriterStat:: 0 FlushedCount 6560000 QueuedCount 49955
2016-04-06T08:18:53.488+05:30 [Info] logWriterStat:: 0 FlushedCount 6570000 QueuedCount 49966
2016-04-06T08:18:53.562+05:30 [Info] logWriterStat:: 0 FlushedCount 6580000 QueuedCount 49843
2016-04-06T08:18:53.629+05:30 [Info] logWriterStat:: 0 FlushedCount 6590000 QueuedCount 49943
2016-04-06T08:18:53.7+05:30 [Info] logWriterStat:: 0 FlushedCount 6600000 QueuedCount 49968
2016-04-06T08:18:53.793+05:30 [Info] logWriterStat:: 0 FlushedCount 6610000 QueuedCount 49974
2016-04-06T08:18:53.865+05:30 [Info] logWriterStat:: 0 FlushedCount 6620000 QueuedCount 49958
2016-04-06T08:18:53.935+05:30 [Info] logWriterStat:: 0 FlushedCount 6630000 QueuedCount 49988
2016-04-06T08:18:54.013+05:30 [Info] logWriterStat:: 0 FlushedCount 6640000 QueuedCount 49942
2016-04-06T08:18:54.106+05:30 [Info] logWriterStat:: 0 FlushedCount 6650000 QueuedCount 49823
2016-04-06T08:18:54.183+05:30 [Info] logWriterStat:: 0 FlushedCount 6660000 QueuedCount 49954
2016-04-06T08:18:54.255+05:30 [Info] logWriterStat:: 0 FlushedCount 6670000 QueuedCount 49978
2016-04-06T08:18:54.326+05:30 [Info] logWriterStat:: 0 FlushedCount 6680000 QueuedCount 49953
2016-04-06T08:18:54.396+05:30 [Info] logWriterStat:: 0 FlushedCount 6690000 QueuedCount 49845
2016-04-06T08:18:54.479+05:30 [Info] logWriterStat:: 0 FlushedCount 6700000 QueuedCount 49932
2016-04-06T08:18:54.553+05:30 [Info] logWriterStat:: 0 FlushedCount 6710000 QueuedCount 49999
2016-04-06T08:18:54.625+05:30 [Info] logWriterStat:: 0 FlushedCount 6720000 QueuedCount 49999
2016-04-06T08:18:54.698+05:30 [Info] logWriterStat:: 0 FlushedCount 6730000 QueuedCount 49981
2016-04-06T08:18:54.768+05:30 [Info] logWriterStat:: 0 FlushedCount 6740000 QueuedCount 49944
2016-04-06T08:18:54.84+05:30 [Info] logWriterStat:: 0 FlushedCount 6750000 QueuedCount 49945
2016-04-06T08:18:54.913+05:30 [Info] logWriterStat:: 0 FlushedCount 6760000 QueuedCount 49992
2016-04-06T08:18:54.986+05:30 [Info] logWriterStat:: 0 FlushedCount 6770000 QueuedCount 49954
2016-04-06T08:18:55.057+05:30 [Info] logWriterStat:: 0 FlushedCount 6780000 QueuedCount 49863
2016-04-06T08:18:55.133+05:30 [Info] logWriterStat:: 0 FlushedCount 6790000 QueuedCount 49925
2016-04-06T08:18:55.214+05:30 [Info] logWriterStat:: 0 FlushedCount 6800000 QueuedCount 49942
2016-04-06T08:18:55.285+05:30 [Info] logWriterStat:: 0 FlushedCount 6810000 QueuedCount 49917
2016-04-06T08:18:55.356+05:30 [Info] logWriterStat:: 0 FlushedCount 6820000 QueuedCount 49924
2016-04-06T08:18:55.435+05:30 [Info] logWriterStat:: 0 FlushedCount 6830000 QueuedCount 49921
2016-04-06T08:18:55.523+05:30 [Info] logWriterStat:: 0 FlushedCount 6840000 QueuedCount 49952
2016-04-06T08:18:55.602+05:30 [Info] logWriterStat:: 0 FlushedCount 6850000 QueuedCount 49958
2016-04-06T08:18:55.687+05:30 [Info] logWriterStat:: 0 FlushedCount 6860000 QueuedCount 49952
2016-04-06T08:18:55.78+05:30 [Info] logWriterStat:: 0 FlushedCount 6870000 QueuedCount 49996
2016-04-06T08:18:55.857+05:30 [Info] logWriterStat:: 0 FlushedCount 6880000 QueuedCount 49329
2016-04-06T08:18:55.929+05:30 [Info] logWriterStat:: 0 FlushedCount 6890000 QueuedCount 43697
2016-04-06T08:18:56.007+05:30 [Info] logWriterStat:: 0 FlushedCount 6900000 QueuedCount 37972
2016-04-06T08:18:56.08+05:30 [Info] logWriterStat:: 0 FlushedCount 6910000 QueuedCount 39387
2016-04-06T08:18:56.159+05:30 [Info] logWriterStat:: 0 FlushedCount 6920000 QueuedCount 36506
2016-04-06T08:18:56.236+05:30 [Info] logWriterStat:: 0 FlushedCount 6930000 QueuedCount 38636
2016-04-06T08:18:56.311+05:30 [Info] logWriterStat:: 0 FlushedCount 6940000 QueuedCount 41734
2016-04-06T08:18:56.382+05:30 [Info] logWriterStat:: 0 FlushedCount 6950000 QueuedCount 39708
2016-04-06T08:18:56.467+05:30 [Info] logWriterStat:: 0 FlushedCount 6960000 QueuedCount 49979
2016-04-06T08:18:56.549+05:30 [Info] logWriterStat:: 0 FlushedCount 6970000 QueuedCount 49942
2016-04-06T08:18:56.621+05:30 [Info] logWriterStat:: 0 FlushedCount 6980000 QueuedCount 49977
2016-04-06T08:18:56.698+05:30 [Info] logWriterStat:: 0 FlushedCount 6990000 QueuedCount 49962
2016-04-06T08:18:56.775+05:30 [Info] logWriterStat:: 0 FlushedCount 7000000 QueuedCount 49926
2016-04-06T08:18:56.851+05:30 [Info] logWriterStat:: 0 FlushedCount 7010000 QueuedCount 49898
2016-04-06T08:18:56.943+05:30 [Info] logWriterStat:: 0 FlushedCount 7020000 QueuedCount 49895
2016-04-06T08:18:57.052+05:30 [Info] logWriterStat:: 0 FlushedCount 7030000 QueuedCount 49932
2016-04-06T08:18:57.129+05:30 [Info] logWriterStat:: 0 FlushedCount 7040000 QueuedCount 49958
2016-04-06T08:18:57.242+05:30 [Info] logWriterStat:: 0 FlushedCount 7050000 QueuedCount 49935
2016-04-06T08:18:57.318+05:30 [Info] logWriterStat:: 0 FlushedCount 7060000 QueuedCount 49967
2016-04-06T08:18:57.393+05:30 [Info] logWriterStat:: 0 FlushedCount 7070000 QueuedCount 49858
2016-04-06T08:18:57.477+05:30 [Info] logWriterStat:: 0 FlushedCount 7080000 QueuedCount 49919
2016-04-06T08:18:57.559+05:30 [Info] logWriterStat:: 0 FlushedCount 7090000 QueuedCount 49990
2016-04-06T08:18:57.639+05:30 [Info] logWriterStat:: 0 FlushedCount 7100000 QueuedCount 49941
2016-04-06T08:18:57.716+05:30 [Info] logWriterStat:: 0 FlushedCount 7110000 QueuedCount 49776
2016-04-06T08:18:57.797+05:30 [Info] logWriterStat:: 0 FlushedCount 7120000 QueuedCount 49973
2016-04-06T08:18:57.871+05:30 [Info] logWriterStat:: 0 FlushedCount 7130000 QueuedCount 49926
2016-04-06T08:18:57.952+05:30 [Info] logWriterStat:: 0 FlushedCount 7140000 QueuedCount 49982
2016-04-06T08:18:58.031+05:30 [Info] logWriterStat:: 0 FlushedCount 7150000 QueuedCount 49942
2016-04-06T08:18:58.108+05:30 [Info] logWriterStat:: 0 FlushedCount 7160000 QueuedCount 49950
2016-04-06T08:18:58.183+05:30 [Info] logWriterStat:: 0 FlushedCount 7170000 QueuedCount 49966
2016-04-06T08:18:58.261+05:30 [Info] logWriterStat:: 0 FlushedCount 7180000 QueuedCount 49928
2016-04-06T08:18:58.337+05:30 [Info] logWriterStat:: 0 FlushedCount 7190000 QueuedCount 49947
2016-04-06T08:18:58.408+05:30 [Info] logWriterStat:: 0 FlushedCount 7200000 QueuedCount 49970
2016-04-06T08:18:58.494+05:30 [Info] logWriterStat:: 0 FlushedCount 7210000 QueuedCount 49984
2016-04-06T08:18:58.568+05:30 [Info] logWriterStat:: 0 FlushedCount 7220000 QueuedCount 49959
2016-04-06T08:18:58.643+05:30 [Info] logWriterStat:: 0 FlushedCount 7230000 QueuedCount 49969
2016-04-06T08:18:58.721+05:30 [Info] logWriterStat:: 0 FlushedCount 7240000 QueuedCount 49970
2016-04-06T08:18:58.796+05:30 [Info] logWriterStat:: 0 FlushedCount 7250000 QueuedCount 49974
2016-04-06T08:18:58.868+05:30 [Info] logWriterStat:: 0 FlushedCount 7260000 QueuedCount 49967
2016-04-06T08:18:58.95+05:30 [Info] logWriterStat:: 0 FlushedCount 7270000 QueuedCount 49950
2016-04-06T08:18:59.03+05:30 [Info] logWriterStat:: 0 FlushedCount 7280000 QueuedCount 49917
2016-04-06T08:18:59.117+05:30 [Info] logWriterStat:: 0 FlushedCount 7290000 QueuedCount 49967
2016-04-06T08:18:59.191+05:30 [Info] logWriterStat:: 0 FlushedCount 7300000 QueuedCount 49890
2016-04-06T08:18:59.267+05:30 [Info] logWriterStat:: 0 FlushedCount 7310000 QueuedCount 49951
2016-04-06T08:18:59.337+05:30 [Info] logWriterStat:: 0 FlushedCount 7320000 QueuedCount 49925
2016-04-06T08:18:59.409+05:30 [Info] logWriterStat:: 0 FlushedCount 7330000 QueuedCount 49932
2016-04-06T08:18:59.491+05:30 [Info] logWriterStat:: 0 FlushedCount 7340000 QueuedCount 49979
2016-04-06T08:18:59.564+05:30 [Info] logWriterStat:: 0 FlushedCount 7350000 QueuedCount 49887
2016-04-06T08:18:59.637+05:30 [Info] logWriterStat:: 0 FlushedCount 7360000 QueuedCount 49919
2016-04-06T08:18:59.706+05:30 [Info] logWriterStat:: 0 FlushedCount 7370000 QueuedCount 49019
2016-04-06T08:18:59.769+05:30 [Info] logWriterStat:: 0 FlushedCount 7380000 QueuedCount 49964
2016-04-06T08:18:59.844+05:30 [Info] logWriterStat:: 0 FlushedCount 7390000 QueuedCount 49981
2016-04-06T08:18:59.918+05:30 [Info] logWriterStat:: 0 FlushedCount 7400000 QueuedCount 49896
2016-04-06T08:18:59.99+05:30 [Info] logWriterStat:: 0 FlushedCount 7410000 QueuedCount 49991
2016-04-06T08:19:00.063+05:30 [Info] logWriterStat:: 0 FlushedCount 7420000 QueuedCount 49959
2016-04-06T08:19:00.137+05:30 [Info] logWriterStat:: 0 FlushedCount 7430000 QueuedCount 49927
2016-04-06T08:19:00.21+05:30 [Info] logWriterStat:: 0 FlushedCount 7440000 QueuedCount 49928
2016-04-06T08:19:00.283+05:30 [Info] logWriterStat:: 0 FlushedCount 7450000 QueuedCount 49993
2016-04-06T08:19:00.356+05:30 [Info] logWriterStat:: 0 FlushedCount 7460000 QueuedCount 49966
2016-04-06T08:19:00.427+05:30 [Info] logWriterStat:: 0 FlushedCount 7470000 QueuedCount 49967
2016-04-06T08:19:00.55+05:30 [Info] logWriterStat:: 0 FlushedCount 7480000 QueuedCount 49996
2016-04-06T08:19:00.625+05:30 [Info] logWriterStat:: 0 FlushedCount 7490000 QueuedCount 49950
2016-04-06T08:19:00.705+05:30 [Info] logWriterStat:: 0 FlushedCount 7500000 QueuedCount 49950
2016-04-06T08:19:00.791+05:30 [Info] logWriterStat:: 0 FlushedCount 7510000 QueuedCount 49981
2016-04-06T08:19:01.005+05:30 [Info] logWriterStat:: 0 FlushedCount 7520000 QueuedCount 48953
2016-04-06T08:19:01.198+05:30 [Info] logWriterStat:: 0 FlushedCount 7530000 QueuedCount 44599
2016-04-06T08:19:01.392+05:30 [Info] logWriterStat:: 0 FlushedCount 7540000 QueuedCount 49840
2016-04-06T08:19:01.511+05:30 [Info] logWriterStat:: 0 FlushedCount 7550000 QueuedCount 49939
2016-04-06T08:19:01.623+05:30 [Info] logWriterStat:: 0 FlushedCount 7560000 QueuedCount 49999
2016-04-06T08:19:01.747+05:30 [Info] logWriterStat:: 0 FlushedCount 7570000 QueuedCount 49936
2016-04-06T08:19:01.875+05:30 [Info] logWriterStat:: 0 FlushedCount 7580000 QueuedCount 49928
2016-04-06T08:19:01.995+05:30 [Info] logWriterStat:: 0 FlushedCount 7590000 QueuedCount 49647
2016-04-06T08:19:02.114+05:30 [Info] logWriterStat:: 0 FlushedCount 7600000 QueuedCount 49908
2016-04-06T08:19:02.266+05:30 [Info] logWriterStat:: 0 FlushedCount 7610000 QueuedCount 49989
2016-04-06T08:19:02.407+05:30 [Info] logWriterStat:: 0 FlushedCount 7620000 QueuedCount 49984
2016-04-06T08:19:02.511+05:30 [Info] logWriterStat:: 0 FlushedCount 7630000 QueuedCount 49989
2016-04-06T08:19:02.582+05:30 [Info] logWriterStat:: 0 FlushedCount 7640000 QueuedCount 49996
2016-04-06T08:19:02.657+05:30 [Info] logWriterStat:: 0 FlushedCount 7650000 QueuedCount 49982
2016-04-06T08:19:02.73+05:30 [Info] logWriterStat:: 0 FlushedCount 7660000 QueuedCount 49980
2016-04-06T08:19:02.803+05:30 [Info] logWriterStat:: 0 FlushedCount 7670000 QueuedCount 49966
2016-04-06T08:19:02.876+05:30 [Info] logWriterStat:: 0 FlushedCount 7680000 QueuedCount 49945
2016-04-06T08:19:02.951+05:30 [Info] logWriterStat:: 0 FlushedCount 7690000 QueuedCount 49873
2016-04-06T08:19:03.029+05:30 [Info] logWriterStat:: 0 FlushedCount 7700000 QueuedCount 49885
2016-04-06T08:19:03.105+05:30 [Info] logWriterStat:: 0 FlushedCount 7710000 QueuedCount 49945
2016-04-06T08:19:03.18+05:30 [Info] logWriterStat:: 0 FlushedCount 7720000 QueuedCount 49993
2016-04-06T08:19:03.253+05:30 [Info] logWriterStat:: 0 FlushedCount 7730000 QueuedCount 49992
2016-04-06T08:19:03.322+05:30 [Info] logWriterStat:: 0 FlushedCount 7740000 QueuedCount 49941
2016-04-06T08:19:03.396+05:30 [Info] logWriterStat:: 0 FlushedCount 7750000 QueuedCount 49952
2016-04-06T08:19:03.477+05:30 [Info] logWriterStat:: 0 FlushedCount 7760000 QueuedCount 49998
2016-04-06T08:19:03.548+05:30 [Info] logWriterStat:: 0 FlushedCount 7770000 QueuedCount 49904
2016-04-06T08:19:03.62+05:30 [Info] logWriterStat:: 0 FlushedCount 7780000 QueuedCount 49967
2016-04-06T08:19:03.707+05:30 [Info] logWriterStat:: 0 FlushedCount 7790000 QueuedCount 49817
2016-04-06T08:19:03.788+05:30 [Info] logWriterStat:: 0 FlushedCount 7800000 QueuedCount 49932
2016-04-06T08:19:03.861+05:30 [Info] logWriterStat:: 0 FlushedCount 7810000 QueuedCount 49990
2016-04-06T08:19:03.934+05:30 [Info] logWriterStat:: 0 FlushedCount 7820000 QueuedCount 49902
2016-04-06T08:19:04.008+05:30 [Info] logWriterStat:: 0 FlushedCount 7830000 QueuedCount 49936
2016-04-06T08:19:04.093+05:30 [Info] logWriterStat:: 0 FlushedCount 7840000 QueuedCount 49957
2016-04-06T08:19:04.164+05:30 [Info] logWriterStat:: 0 FlushedCount 7850000 QueuedCount 49935
2016-04-06T08:19:04.238+05:30 [Info] logWriterStat:: 0 FlushedCount 7860000 QueuedCount 49989
2016-04-06T08:19:04.312+05:30 [Info] logWriterStat:: 0 FlushedCount 7870000 QueuedCount 48906
2016-04-06T08:19:04.375+05:30 [Info] logWriterStat:: 0 FlushedCount 7880000 QueuedCount 48798
2016-04-06T08:19:04.464+05:30 [Info] logWriterStat:: 0 FlushedCount 7890000 QueuedCount 49965
2016-04-06T08:19:04.531+05:30 [Info] logWriterStat:: 0 FlushedCount 7900000 QueuedCount 45054
2016-04-06T08:19:04.601+05:30 [Info] logWriterStat:: 0 FlushedCount 7910000 QueuedCount 49950
2016-04-06T08:19:04.673+05:30 [Info] logWriterStat:: 0 FlushedCount 7920000 QueuedCount 49936
2016-04-06T08:19:04.746+05:30 [Info] logWriterStat:: 0 FlushedCount 7930000 QueuedCount 49975
2016-04-06T08:19:04.815+05:30 [Info] logWriterStat:: 0 FlushedCount 7940000 QueuedCount 49962
2016-04-06T08:19:04.884+05:30 [Info] logWriterStat:: 0 FlushedCount 7950000 QueuedCount 49967
2016-04-06T08:19:04.96+05:30 [Info] logWriterStat:: 0 FlushedCount 7960000 QueuedCount 49919
2016-04-06T08:19:05.032+05:30 [Info] logWriterStat:: 0 FlushedCount 7970000 QueuedCount 49985
2016-04-06T08:19:05.103+05:30 [Info] logWriterStat:: 0 FlushedCount 7980000 QueuedCount 49866
2016-04-06T08:19:05.173+05:30 [Info] logWriterStat:: 0 FlushedCount 7990000 QueuedCount 49986
2016-04-06T08:19:05.243+05:30 [Info] logWriterStat:: 0 FlushedCount 8000000 QueuedCount 49858
2016-04-06T08:19:05.315+05:30 [Info] logWriterStat:: 0 FlushedCount 8010000 QueuedCount 49925
2016-04-06T08:19:05.39+05:30 [Info] logWriterStat:: 0 FlushedCount 8020000 QueuedCount 49935
2016-04-06T08:19:05.464+05:30 [Info] logWriterStat:: 0 FlushedCount 8030000 QueuedCount 48842
2016-04-06T08:19:05.531+05:30 [Info] logWriterStat:: 0 FlushedCount 8040000 QueuedCount 49977
2016-04-06T08:19:05.604+05:30 [Info] logWriterStat:: 0 FlushedCount 8050000 QueuedCount 49990
2016-04-06T08:19:05.674+05:30 [Info] logWriterStat:: 0 FlushedCount 8060000 QueuedCount 49914
2016-04-06T08:19:05.756+05:30 [Info] logWriterStat:: 0 FlushedCount 8070000 QueuedCount 49957
2016-04-06T08:19:05.828+05:30 [Info] logWriterStat:: 0 FlushedCount 8080000 QueuedCount 49968
2016-04-06T08:19:05.9+05:30 [Info] logWriterStat:: 0 FlushedCount 8090000 QueuedCount 49992
2016-04-06T08:19:05.972+05:30 [Info] logWriterStat:: 0 FlushedCount 8100000 QueuedCount 49901
2016-04-06T08:19:06.044+05:30 [Info] logWriterStat:: 0 FlushedCount 8110000 QueuedCount 49939
2016-04-06T08:19:06.115+05:30 [Info] logWriterStat:: 0 FlushedCount 8120000 QueuedCount 49921
2016-04-06T08:19:06.218+05:30 [Info] logWriterStat:: 0 FlushedCount 8130000 QueuedCount 49986
2016-04-06T08:19:06.318+05:30 [Info] logWriterStat:: 0 FlushedCount 8140000 QueuedCount 49917
2016-04-06T08:19:06.394+05:30 [Info] logWriterStat:: 0 FlushedCount 8150000 QueuedCount 49965
2016-04-06T08:19:06.473+05:30 [Info] logWriterStat:: 0 FlushedCount 8160000 QueuedCount 49981
2016-04-06T08:19:06.545+05:30 [Info] logWriterStat:: 0 FlushedCount 8170000 QueuedCount 49865
2016-04-06T08:19:06.615+05:30 [Info] logWriterStat:: 0 FlushedCount 8180000 QueuedCount 49998
2016-04-06T08:19:06.692+05:30 [Info] logWriterStat:: 0 FlushedCount 8190000 QueuedCount 49939
2016-04-06T08:19:06.765+05:30 [Info] logWriterStat:: 0 FlushedCount 8200000 QueuedCount 49916
2016-04-06T08:19:06.834+05:30 [Info] logWriterStat:: 0 FlushedCount 8210000 QueuedCount 49978
2016-04-06T08:19:06.906+05:30 [Info] logWriterStat:: 0 FlushedCount 8220000 QueuedCount 49969
2016-04-06T08:19:06.977+05:30 [Info] logWriterStat:: 0 FlushedCount 8230000 QueuedCount 49889
2016-04-06T08:19:07.049+05:30 [Info] logWriterStat:: 0 FlushedCount 8240000 QueuedCount 49990
2016-04-06T08:19:07.12+05:30 [Info] logWriterStat:: 0 FlushedCount 8250000 QueuedCount 49947
2016-04-06T08:19:07.191+05:30 [Info] logWriterStat:: 0 FlushedCount 8260000 QueuedCount 49949
2016-04-06T08:19:07.274+05:30 [Info] logWriterStat:: 0 FlushedCount 8270000 QueuedCount 49925
2016-04-06T08:19:07.345+05:30 [Info] logWriterStat:: 0 FlushedCount 8280000 QueuedCount 49954
2016-04-06T08:19:07.413+05:30 [Info] logWriterStat:: 0 FlushedCount 8290000 QueuedCount 49920
2016-04-06T08:19:07.493+05:30 [Info] logWriterStat:: 0 FlushedCount 8300000 QueuedCount 49972
2016-04-06T08:19:07.562+05:30 [Info] logWriterStat:: 0 FlushedCount 8310000 QueuedCount 49606
2016-04-06T08:19:07.631+05:30 [Info] logWriterStat:: 0 FlushedCount 8320000 QueuedCount 49974
2016-04-06T08:19:07.703+05:30 [Info] logWriterStat:: 0 FlushedCount 8330000 QueuedCount 49910
2016-04-06T08:19:07.778+05:30 [Info] logWriterStat:: 0 FlushedCount 8340000 QueuedCount 49964
2016-04-06T08:19:07.846+05:30 [Info] logWriterStat:: 0 FlushedCount 8350000 QueuedCount 49959
2016-04-06T08:19:07.916+05:30 [Info] logWriterStat:: 0 FlushedCount 8360000 QueuedCount 49983
2016-04-06T08:19:07.986+05:30 [Info] logWriterStat:: 0 FlushedCount 8370000 QueuedCount 49955
2016-04-06T08:19:08.056+05:30 [Info] logWriterStat:: 0 FlushedCount 8380000 QueuedCount 49961
2016-04-06T08:19:08.121+05:30 [Info] logWriterStat:: 0 FlushedCount 8390000 QueuedCount 48453
2016-04-06T08:19:08.182+05:30 [Info] logWriterStat:: 0 FlushedCount 8400000 QueuedCount 43906
2016-04-06T08:19:08.25+05:30 [Info] logWriterStat:: 0 FlushedCount 8410000 QueuedCount 40070
2016-04-06T08:19:08.314+05:30 [Info] logWriterStat:: 0 FlushedCount 8420000 QueuedCount 40561
2016-04-06T08:19:08.385+05:30 [Info] logWriterStat:: 0 FlushedCount 8430000 QueuedCount 37748
2016-04-06T08:19:08.462+05:30 [Info] logWriterStat:: 0 FlushedCount 8440000 QueuedCount 36513
2016-04-06T08:19:08.529+05:30 [Info] logWriterStat:: 0 FlushedCount 8450000 QueuedCount 37464
2016-04-06T08:19:08.599+05:30 [Info] logWriterStat:: 0 FlushedCount 8460000 QueuedCount 42143
2016-04-06T08:19:08.669+05:30 [Info] logWriterStat:: 0 FlushedCount 8470000 QueuedCount 41736
2016-04-06T08:19:08.737+05:30 [Info] logWriterStat:: 0 FlushedCount 8480000 QueuedCount 39559
2016-04-06T08:19:08.804+05:30 [Info] logWriterStat:: 0 FlushedCount 8490000 QueuedCount 39313
2016-04-06T08:19:08.869+05:30 [Info] logWriterStat:: 0 FlushedCount 8500000 QueuedCount 39139
2016-04-06T08:19:08.937+05:30 [Info] logWriterStat:: 0 FlushedCount 8510000 QueuedCount 39454
2016-04-06T08:19:09.01+05:30 [Info] logWriterStat:: 0 FlushedCount 8520000 QueuedCount 41894
2016-04-06T08:19:09.086+05:30 [Info] logWriterStat:: 0 FlushedCount 8530000 QueuedCount 42450
2016-04-06T08:19:09.152+05:30 [Info] logWriterStat:: 0 FlushedCount 8540000 QueuedCount 43010
2016-04-06T08:19:09.216+05:30 [Info] logWriterStat:: 0 FlushedCount 8550000 QueuedCount 38463
2016-04-06T08:19:09.285+05:30 [Info] logWriterStat:: 0 FlushedCount 8560000 QueuedCount 37872
2016-04-06T08:19:09.357+05:30 [Info] logWriterStat:: 0 FlushedCount 8570000 QueuedCount 39268
2016-04-06T08:19:09.423+05:30 [Info] logWriterStat:: 0 FlushedCount 8580000 QueuedCount 39183
2016-04-06T08:19:09.499+05:30 [Info] logWriterStat:: 0 FlushedCount 8590000 QueuedCount 34054
2016-04-06T08:19:09.564+05:30 [Info] logWriterStat:: 0 FlushedCount 8600000 QueuedCount 42237
2016-04-06T08:19:09.637+05:30 [Info] logWriterStat:: 0 FlushedCount 8610000 QueuedCount 41174
2016-04-06T08:19:09.71+05:30 [Info] logWriterStat:: 0 FlushedCount 8620000 QueuedCount 37158
2016-04-06T08:19:09.78+05:30 [Info] logWriterStat:: 0 FlushedCount 8630000 QueuedCount 42015
2016-04-06T08:19:09.846+05:30 [Info] logWriterStat:: 0 FlushedCount 8640000 QueuedCount 39227
2016-04-06T08:19:09.915+05:30 [Info] logWriterStat:: 0 FlushedCount 8650000 QueuedCount 36433
2016-04-06T08:19:09.98+05:30 [Info] logWriterStat:: 0 FlushedCount 8660000 QueuedCount 33266
2016-04-06T08:19:10.047+05:30 [Info] logWriterStat:: 0 FlushedCount 8670000 QueuedCount 32136
2016-04-06T08:19:10.116+05:30 [Info] logWriterStat:: 0 FlushedCount 8680000 QueuedCount 38107
2016-04-06T08:19:10.185+05:30 [Info] logWriterStat:: 0 FlushedCount 8690000 QueuedCount 40701
2016-04-06T08:19:10.252+05:30 [Info] logWriterStat:: 0 FlushedCount 8700000 QueuedCount 39039
2016-04-06T08:19:10.32+05:30 [Info] logWriterStat:: 0 FlushedCount 8710000 QueuedCount 39193
2016-04-06T08:19:10.39+05:30 [Info] logWriterStat:: 0 FlushedCount 8720000 QueuedCount 34922
2016-04-06T08:19:10.485+05:30 [Info] logWriterStat:: 0 FlushedCount 8730000 QueuedCount 49972
2016-04-06T08:19:10.591+05:30 [Info] logWriterStat:: 0 FlushedCount 8740000 QueuedCount 49940
2016-04-06T08:19:10.663+05:30 [Info] logWriterStat:: 0 FlushedCount 8750000 QueuedCount 49912
2016-04-06T08:19:10.733+05:30 [Info] logWriterStat:: 0 FlushedCount 8760000 QueuedCount 49905
2016-04-06T08:19:10.816+05:30 [Info] logWriterStat:: 0 FlushedCount 8770000 QueuedCount 49980
2016-04-06T08:19:10.887+05:30 [Info] logWriterStat:: 0 FlushedCount 8780000 QueuedCount 49945
2016-04-06T08:19:10.962+05:30 [Info] logWriterStat:: 0 FlushedCount 8790000 QueuedCount 49956
2016-04-06T08:19:11.034+05:30 [Info] logWriterStat:: 0 FlushedCount 8800000 QueuedCount 49979
2016-04-06T08:19:11.107+05:30 [Info] logWriterStat:: 0 FlushedCount 8810000 QueuedCount 49974
2016-04-06T08:19:11.179+05:30 [Info] logWriterStat:: 0 FlushedCount 8820000 QueuedCount 49983
2016-04-06T08:19:11.248+05:30 [Info] logWriterStat:: 0 FlushedCount 8830000 QueuedCount 49927
2016-04-06T08:19:11.315+05:30 [Info] logWriterStat:: 0 FlushedCount 8840000 QueuedCount 49934
2016-04-06T08:19:11.384+05:30 [Info] logWriterStat:: 0 FlushedCount 8850000 QueuedCount 49960
2016-04-06T08:19:11.466+05:30 [Info] logWriterStat:: 0 FlushedCount 8860000 QueuedCount 49978
2016-04-06T08:19:11.537+05:30 [Info] logWriterStat:: 0 FlushedCount 8870000 QueuedCount 49997
2016-04-06T08:19:11.611+05:30 [Info] logWriterStat:: 0 FlushedCount 8880000 QueuedCount 49940
2016-04-06T08:19:11.683+05:30 [Info] logWriterStat:: 0 FlushedCount 8890000 QueuedCount 49829
2016-04-06T08:19:11.758+05:30 [Info] logWriterStat:: 0 FlushedCount 8900000 QueuedCount 49835
2016-04-06T08:19:11.828+05:30 [Info] logWriterStat:: 0 FlushedCount 8910000 QueuedCount 49916
2016-04-06T08:19:11.896+05:30 [Info] logWriterStat:: 0 FlushedCount 8920000 QueuedCount 49986
2016-04-06T08:19:11.97+05:30 [Info] logWriterStat:: 0 FlushedCount 8930000 QueuedCount 49997
2016-04-06T08:19:12.044+05:30 [Info] logWriterStat:: 0 FlushedCount 8940000 QueuedCount 49978
2016-04-06T08:19:12.115+05:30 [Info] logWriterStat:: 0 FlushedCount 8950000 QueuedCount 49980
2016-04-06T08:19:12.186+05:30 [Info] logWriterStat:: 0 FlushedCount 8960000 QueuedCount 49930
2016-04-06T08:19:12.271+05:30 [Info] logWriterStat:: 0 FlushedCount 8970000 QueuedCount 49969
2016-04-06T08:19:12.343+05:30 [Info] logWriterStat:: 0 FlushedCount 8980000 QueuedCount 49976
2016-04-06T08:19:12.409+05:30 [Info] logWriterStat:: 0 FlushedCount 8990000 QueuedCount 49641
2016-04-06T08:19:12.493+05:30 [Info] logWriterStat:: 0 FlushedCount 9000000 QueuedCount 49953
2016-04-06T08:19:12.566+05:30 [Info] logWriterStat:: 0 FlushedCount 9010000 QueuedCount 49932
2016-04-06T08:19:12.645+05:30 [Info] logWriterStat:: 0 FlushedCount 9020000 QueuedCount 49896
2016-04-06T08:19:12.714+05:30 [Info] logWriterStat:: 0 FlushedCount 9030000 QueuedCount 49952
2016-04-06T08:19:12.786+05:30 [Info] logWriterStat:: 0 FlushedCount 9040000 QueuedCount 49964
2016-04-06T08:19:12.855+05:30 [Info] logWriterStat:: 0 FlushedCount 9050000 QueuedCount 49903
2016-04-06T08:19:12.923+05:30 [Info] logWriterStat:: 0 FlushedCount 9060000 QueuedCount 49883
2016-04-06T08:19:12.997+05:30 [Info] logWriterStat:: 0 FlushedCount 9070000 QueuedCount 49969
2016-04-06T08:19:13.068+05:30 [Info] logWriterStat:: 0 FlushedCount 9080000 QueuedCount 49870
2016-04-06T08:19:13.14+05:30 [Info] logWriterStat:: 0 FlushedCount 9090000 QueuedCount 49984
2016-04-06T08:19:13.211+05:30 [Info] logWriterStat:: 0 FlushedCount 9100000 QueuedCount 49987
2016-04-06T08:19:13.283+05:30 [Info] logWriterStat:: 0 FlushedCount 9110000 QueuedCount 49982
2016-04-06T08:19:13.357+05:30 [Info] logWriterStat:: 0 FlushedCount 9120000 QueuedCount 49991
2016-04-06T08:19:13.424+05:30 [Info] logWriterStat:: 0 FlushedCount 9130000 QueuedCount 49843
2016-04-06T08:19:13.506+05:30 [Info] logWriterStat:: 0 FlushedCount 9140000 QueuedCount 48887
2016-04-06T08:19:13.579+05:30 [Info] logWriterStat:: 0 FlushedCount 9150000 QueuedCount 47993
2016-04-06T08:19:13.649+05:30 [Info] logWriterStat:: 0 FlushedCount 9160000 QueuedCount 47183
2016-04-06T08:19:13.725+05:30 [Info] logWriterStat:: 0 FlushedCount 9170000 QueuedCount 46862
2016-04-06T08:19:13.799+05:30 [Info] logWriterStat:: 0 FlushedCount 9180000 QueuedCount 46855
2016-04-06T08:19:13.87+05:30 [Info] logWriterStat:: 0 FlushedCount 9190000 QueuedCount 46835
2016-04-06T08:19:13.952+05:30 [Info] logWriterStat:: 0 FlushedCount 9200000 QueuedCount 46790
2016-04-06T08:19:14.029+05:30 [Info] logWriterStat:: 0 FlushedCount 9210000 QueuedCount 46823
2016-04-06T08:19:14.118+05:30 [Info] logWriterStat:: 0 FlushedCount 9220000 QueuedCount 46787
2016-04-06T08:19:14.191+05:30 [Info] logWriterStat:: 0 FlushedCount 9230000 QueuedCount 46169
2016-04-06T08:19:14.263+05:30 [Info] logWriterStat:: 0 FlushedCount 9240000 QueuedCount 45554
2016-04-06T08:19:14.335+05:30 [Info] logWriterStat:: 0 FlushedCount 9250000 QueuedCount 44976
2016-04-06T08:19:14.407+05:30 [Info] logWriterStat:: 0 FlushedCount 9260000 QueuedCount 44321
2016-04-06T08:19:14.488+05:30 [Info] logWriterStat:: 0 FlushedCount 9270000 QueuedCount 43727
2016-04-06T08:19:14.561+05:30 [Info] logWriterStat:: 0 FlushedCount 9280000 QueuedCount 43721
2016-04-06T08:19:14.63+05:30 [Info] logWriterStat:: 0 FlushedCount 9290000 QueuedCount 43678
2016-04-06T08:19:14.695+05:30 [Info] logWriterStat:: 0 FlushedCount 9300000 QueuedCount 43727
2016-04-06T08:19:14.765+05:30 [Info] logWriterStat:: 0 FlushedCount 9310000 QueuedCount 43739
2016-04-06T08:19:14.832+05:30 [Info] logWriterStat:: 0 FlushedCount 9320000 QueuedCount 43727
2016-04-06T08:19:14.905+05:30 [Info] logWriterStat:: 0 FlushedCount 9330000 QueuedCount 43679
2016-04-06T08:19:14.98+05:30 [Info] logWriterStat:: 0 FlushedCount 9340000 QueuedCount 43733
2016-04-06T08:19:15.054+05:30 [Info] logWriterStat:: 0 FlushedCount 9350000 QueuedCount 43739
2016-04-06T08:19:15.128+05:30 [Info] logWriterStat:: 0 FlushedCount 9360000 QueuedCount 43688
2016-04-06T08:19:15.2+05:30 [Info] logWriterStat:: 0 FlushedCount 9370000 QueuedCount 43718
2016-04-06T08:19:15.275+05:30 [Info] logWriterStat:: 0 FlushedCount 9380000 QueuedCount 43711
2016-04-06T08:19:15.349+05:30 [Info] logWriterStat:: 0 FlushedCount 9390000 QueuedCount 43733
2016-04-06T08:19:15.421+05:30 [Info] logWriterStat:: 0 FlushedCount 9400000 QueuedCount 43719
2016-04-06T08:19:15.503+05:30 [Info] logWriterStat:: 0 FlushedCount 9410000 QueuedCount 43715
2016-04-06T08:19:15.582+05:30 [Info] logWriterStat:: 0 FlushedCount 9420000 QueuedCount 43727
2016-04-06T08:19:15.657+05:30 [Info] logWriterStat:: 0 FlushedCount 9430000 QueuedCount 43721
2016-04-06T08:19:15.746+05:30 [Info] logWriterStat:: 0 FlushedCount 9440000 QueuedCount 43727
2016-04-06T08:19:15.823+05:30 [Info] logWriterStat:: 0 FlushedCount 9450000 QueuedCount 43711
2016-04-06T08:19:15.895+05:30 [Info] logWriterStat:: 0 FlushedCount 9460000 QueuedCount 43731
2016-04-06T08:19:15.967+05:30 [Info] logWriterStat:: 0 FlushedCount 9470000 QueuedCount 43650
2016-04-06T08:19:16.04+05:30 [Info] logWriterStat:: 0 FlushedCount 9480000 QueuedCount 42981
2016-04-06T08:19:16.113+05:30 [Info] logWriterStat:: 0 FlushedCount 9490000 QueuedCount 42157
2016-04-06T08:19:16.186+05:30 [Info] logWriterStat:: 0 FlushedCount 9500000 QueuedCount 41358
2016-04-06T08:19:16.26+05:30 [Info] logWriterStat:: 0 FlushedCount 9510000 QueuedCount 40651
2016-04-06T08:19:16.334+05:30 [Info] logWriterStat:: 0 FlushedCount 9520000 QueuedCount 40580
2016-04-06T08:19:16.405+05:30 [Info] logWriterStat:: 0 FlushedCount 9530000 QueuedCount 40589
2016-04-06T08:19:16.483+05:30 [Info] logWriterStat:: 0 FlushedCount 9540000 QueuedCount 40156
2016-04-06T08:19:16.554+05:30 [Info] logWriterStat:: 0 FlushedCount 9550000 QueuedCount 39612
2016-04-06T08:19:16.621+05:30 [Info] logWriterStat:: 0 FlushedCount 9560000 QueuedCount 37984
2016-04-06T08:19:16.69+05:30 [Info] logWriterStat:: 0 FlushedCount 9570000 QueuedCount 36080
2016-04-06T08:19:16.76+05:30 [Info] logWriterStat:: 0 FlushedCount 9580000 QueuedCount 33616
2016-04-06T08:19:16.825+05:30 [Info] logWriterStat:: 0 FlushedCount 9590000 QueuedCount 32415
2016-04-06T08:19:16.893+05:30 [Info] logWriterStat:: 0 FlushedCount 9600000 QueuedCount 30441
2016-04-06T08:19:16.966+05:30 [Info] logWriterStat:: 0 FlushedCount 9610000 QueuedCount 29415
2016-04-06T08:19:17.119+05:30 [Info] logWriterStat:: 0 FlushedCount 9620000 QueuedCount 28013
2016-04-06T08:19:17.19+05:30 [Info] logWriterStat:: 0 FlushedCount 9630000 QueuedCount 28064
2016-04-06T08:19:17.317+05:30 [Info] logWriterStat:: 0 FlushedCount 9640000 QueuedCount 26117
2016-04-06T08:19:17.455+05:30 [Info] logWriterStat:: 0 FlushedCount 9650000 QueuedCount 23091
2016-04-06T08:19:17.559+05:30 [Info] logWriterStat:: 0 FlushedCount 9660000 QueuedCount 19963
2016-04-06T08:19:17.673+05:30 [Info] logWriterStat:: 0 FlushedCount 9670000 QueuedCount 16080
2016-04-06T08:19:17.781+05:30 [Info] logWriterStat:: 0 FlushedCount 9680000 QueuedCount 12305
2016-04-06T08:19:17.849+05:30 [Info] logWriterStat:: 0 FlushedCount 9690000 QueuedCount 6291
2016-04-06T08:19:17.971+05:30 [Info] logWriterStat:: 0 FlushedCount 9700000 QueuedCount 1441
2016-04-06T08:19:18.002+05:30 [Info] MemDBSlice::OpenSnapshot SliceId 0 IndexInstId 0 Creating New Snapshot SnapshotInfo: count:9701441 committed:false
2016-04-06T08:19:18.192+05:30 [Info] logWriterStat:: 0 FlushedCount 9710000 QueuedCount 46777
2016-04-06T08:19:18.324+05:30 [Info] logWriterStat:: 0 FlushedCount 9720000 QueuedCount 46590
2016-04-06T08:19:18.52+05:30 [Info] logWriterStat:: 0 FlushedCount 9730000 QueuedCount 44478
2016-04-06T08:19:18.684+05:30 [Info] logWriterStat:: 0 FlushedCount 9740000 QueuedCount 46460
2016-04-06T08:19:18.832+05:30 [Info] logWriterStat:: 0 FlushedCount 9750000 QueuedCount 45816
2016-04-06T08:19:19.007+05:30 [Info] logWriterStat:: 0 FlushedCount 9760000 QueuedCount 46105
2016-04-06T08:19:19.166+05:30 [Info] logWriterStat:: 0 FlushedCount 9770000 QueuedCount 45015
2016-04-06T08:19:19.351+05:30 [Info] logWriterStat:: 0 FlushedCount 9780000 QueuedCount 45100
2016-04-06T08:19:19.59+05:30 [Info] logWriterStat:: 0 FlushedCount 9790000 QueuedCount 44268
2016-04-06T08:19:19.951+05:30 [Info] logWriterStat:: 0 FlushedCount 9800000 QueuedCount 41878
2016-04-06T08:19:20.048+05:30 [Info] logWriterStat:: 0 FlushedCount 9810000 QueuedCount 41491
2016-04-06T08:19:20.186+05:30 [Info] logWriterStat:: 0 FlushedCount 9820000 QueuedCount 39614
2016-04-06T08:19:20.324+05:30 [Info] logWriterStat:: 0 FlushedCount 9830000 QueuedCount 39275
2016-04-06T08:19:20.506+05:30 [Info] logWriterStat:: 0 FlushedCount 9840000 QueuedCount 40586
2016-04-06T08:19:20.688+05:30 [Info] logWriterStat:: 0 FlushedCount 9850000 QueuedCount 39715
2016-04-06T08:19:20.882+05:30 [Info] logWriterStat:: 0 FlushedCount 9860000 QueuedCount 38955
2016-04-06T08:19:21.054+05:30 [Info] logWriterStat:: 0 FlushedCount 9870000 QueuedCount 38409
2016-04-06T08:19:21.267+05:30 [Info] logWriterStat:: 0 FlushedCount 9880000 QueuedCount 36758
2016-04-06T08:19:21.414+05:30 [Info] logWriterStat:: 0 FlushedCount 9890000 QueuedCount 35834
2016-04-06T08:19:21.61+05:30 [Info] logWriterStat:: 0 FlushedCount 9900000 QueuedCount 32196
2016-04-06T08:19:21.86+05:30 [Info] logWriterStat:: 0 FlushedCount 9910000 QueuedCount 29300
2016-04-06T08:19:22.126+05:30 [Info] logWriterStat:: 0 FlushedCount 9920000 QueuedCount 26402
2016-04-06T08:19:22.42+05:30 [Info] logWriterStat:: 0 FlushedCount 9930000 QueuedCount 23116
2016-04-06T08:19:22.786+05:30 [Info] logWriterStat:: 0 FlushedCount 9940000 QueuedCount 22052
2016-04-06T08:19:23.073+05:30 [Info] logWriterStat:: 0 FlushedCount 9950000 QueuedCount 20168
2016-04-06T08:19:23.306+05:30 [Info] logWriterStat:: 0 FlushedCount 9960000 QueuedCount 15122
2016-04-06T08:19:23.51+05:30 [Info] logWriterStat:: 0 FlushedCount 9970000 QueuedCount 8652
2016-04-06T08:19:23.921+05:30 [Info] logWriterStat:: 0 FlushedCount 9980000 QueuedCount 5651
2016-04-06T08:19:25.015+05:30 [Info] logWriterStat:: 0 FlushedCount 9990000 QueuedCount 3095
2016-04-06T08:19:26.481+05:30 [Info] logWriterStat:: 0 FlushedCount 10000000 QueuedCount 0
2016-04-06T08:19:26.483+05:30 [Info] MemDBSlice::OpenSnapshot SliceId 0 IndexInstId 0 Creating New Snapshot SnapshotInfo: count:10000000 committed:false
2016-04-06T08:19:26.745+05:30 [Info] logWriterStat:: 0 FlushedCount 10010000 QueuedCount 49758
2016-04-06T08:19:26.913+05:30 [Info] logWriterStat:: 0 FlushedCount 10020000 QueuedCount 49027
2016-04-06T08:19:27.05+05:30 [Info] logWriterStat:: 0 FlushedCount 10030000 QueuedCount 47408
2016-04-06T08:19:27.221+05:30 [Info] logWriterStat:: 0 FlushedCount 10040000 QueuedCount 47096
2016-04-06T08:19:27.394+05:30 [Info] logWriterStat:: 0 FlushedCount 10050000 QueuedCount 47550
2016-04-06T08:19:27.586+05:30 [Info] logWriterStat:: 0 FlushedCount 10060000 QueuedCount 48437
2016-04-06T08:19:27.763+05:30 [Info] logWriterStat:: 0 FlushedCount 10070000 QueuedCount 47945
2016-04-06T08:19:27.987+05:30 [Info] logWriterStat:: 0 FlushedCount 10080000 QueuedCount 49873
2016-04-06T08:19:28.142+05:30 [Info] logWriterStat:: 0 FlushedCount 10090000 QueuedCount 47758
2016-04-06T08:19:28.478+05:30 [Info] logWriterStat:: 0 FlushedCount 10100000 QueuedCount 49563
2016-04-06T08:19:29.368+05:30 [Info] logWriterStat:: 0 FlushedCount 10110000 QueuedCount 49813
2016-04-06T08:19:29.613+05:30 [Info] logWriterStat:: 0 FlushedCount 10120000 QueuedCount 47531
2016-04-06T08:19:29.754+05:30 [Info] logWriterStat:: 0 FlushedCount 10130000 QueuedCount 48932
2016-04-06T08:19:29.894+05:30 [Info] logWriterStat:: 0 FlushedCount 10140000 QueuedCount 48504
2016-04-06T08:19:30.119+05:30 [Info] logWriterStat:: 0 FlushedCount 10150000 QueuedCount 49299
2016-04-06T08:19:30.259+05:30 [Info] logWriterStat:: 0 FlushedCount 10160000 QueuedCount 49978
2016-04-06T08:19:30.361+05:30 [Info] logWriterStat:: 0 FlushedCount 10170000 QueuedCount 49922
2016-04-06T08:19:30.465+05:30 [Info] logWriterStat:: 0 FlushedCount 10180000 QueuedCount 49976
2016-04-06T08:19:30.597+05:30 [Info] logWriterStat:: 0 FlushedCount 10190000 QueuedCount 49942
2016-04-06T08:19:30.687+05:30 [Info] logWriterStat:: 0 FlushedCount 10200000 QueuedCount 49996
2016-04-06T08:19:30.776+05:30 [Info] logWriterStat:: 0 FlushedCount 10210000 QueuedCount 49593
2016-04-06T08:19:30.864+05:30 [Info] logWriterStat:: 0 FlushedCount 10220000 QueuedCount 49973
2016-04-06T08:19:30.95+05:30 [Info] logWriterStat:: 0 FlushedCount 10230000 QueuedCount 49992
2016-04-06T08:19:31.039+05:30 [Info] logWriterStat:: 0 FlushedCount 10240000 QueuedCount 49952
2016-04-06T08:19:31.125+05:30 [Info] logWriterStat:: 0 FlushedCount 10250000 QueuedCount 49924
2016-04-06T08:19:31.208+05:30 [Info] logWriterStat:: 0 FlushedCount 10260000 QueuedCount 49970
2016-04-06T08:19:31.295+05:30 [Info] logWriterStat:: 0 FlushedCount 10270000 QueuedCount 49957
2016-04-06T08:19:31.382+05:30 [Info] logWriterStat:: 0 FlushedCount 10280000 QueuedCount 49997
2016-04-06T08:19:31.478+05:30 [Info] logWriterStat:: 0 FlushedCount 10290000 QueuedCount 49985
2016-04-06T08:19:31.559+05:30 [Info] logWriterStat:: 0 FlushedCount 10300000 QueuedCount 49964
2016-04-06T08:19:31.645+05:30 [Info] logWriterStat:: 0 FlushedCount 10310000 QueuedCount 49984
2016-04-06T08:19:31.731+05:30 [Info] logWriterStat:: 0 FlushedCount 10320000 QueuedCount 49991
2016-04-06T08:19:31.82+05:30 [Info] logWriterStat:: 0 FlushedCount 10330000 QueuedCount 49905
2016-04-06T08:19:31.907+05:30 [Info] logWriterStat:: 0 FlushedCount 10340000 QueuedCount 49942
2016-04-06T08:19:31.996+05:30 [Info] logWriterStat:: 0 FlushedCount 10350000 QueuedCount 49972
2016-04-06T08:19:32.083+05:30 [Info] logWriterStat:: 0 FlushedCount 10360000 QueuedCount 49874
2016-04-06T08:19:32.17+05:30 [Info] logWriterStat:: 0 FlushedCount 10370000 QueuedCount 49922
2016-04-06T08:19:32.256+05:30 [Info] logWriterStat:: 0 FlushedCount 10380000 QueuedCount 49984
2016-04-06T08:19:32.336+05:30 [Info] logWriterStat:: 0 FlushedCount 10390000 QueuedCount 49867
2016-04-06T08:19:32.435+05:30 [Info] logWriterStat:: 0 FlushedCount 10400000 QueuedCount 49986
2016-04-06T08:19:32.536+05:30 [Info] logWriterStat:: 0 FlushedCount 10410000 QueuedCount 49998
2016-04-06T08:19:32.62+05:30 [Info] logWriterStat:: 0 FlushedCount 10420000 QueuedCount 49341
2016-04-06T08:19:32.707+05:30 [Info] logWriterStat:: 0 FlushedCount 10430000 QueuedCount 49960
2016-04-06T08:19:32.796+05:30 [Info] logWriterStat:: 0 FlushedCount 10440000 QueuedCount 49978
2016-04-06T08:19:32.871+05:30 [Info] logWriterStat:: 0 FlushedCount 10450000 QueuedCount 49988
2016-04-06T08:19:32.957+05:30 [Info] logWriterStat:: 0 FlushedCount 10460000 QueuedCount 49972
2016-04-06T08:19:33.046+05:30 [Info] logWriterStat:: 0 FlushedCount 10470000 QueuedCount 49934
2016-04-06T08:19:33.132+05:30 [Info] logWriterStat:: 0 FlushedCount 10480000 QueuedCount 49980
2016-04-06T08:19:33.217+05:30 [Info] logWriterStat:: 0 FlushedCount 10490000 QueuedCount 49979
2016-04-06T08:19:33.307+05:30 [Info] logWriterStat:: 0 FlushedCount 10500000 QueuedCount 49909
2016-04-06T08:19:33.396+05:30 [Info] logWriterStat:: 0 FlushedCount 10510000 QueuedCount 49859
2016-04-06T08:19:33.476+05:30 [Info] logWriterStat:: 0 FlushedCount 10520000 QueuedCount 48323
2016-04-06T08:19:33.56+05:30 [Info] logWriterStat:: 0 FlushedCount 10530000 QueuedCount 49975
2016-04-06T08:19:33.648+05:30 [Info] logWriterStat:: 0 FlushedCount 10540000 QueuedCount 49883
2016-04-06T08:19:33.731+05:30 [Info] logWriterStat:: 0 FlushedCount 10550000 QueuedCount 49984
2016-04-06T08:19:33.827+05:30 [Info] logWriterStat:: 0 FlushedCount 10560000 QueuedCount 49988
2016-04-06T08:19:33.915+05:30 [Info] logWriterStat:: 0 FlushedCount 10570000 QueuedCount 49965
2016-04-06T08:19:34.006+05:30 [Info] logWriterStat:: 0 FlushedCount 10580000 QueuedCount 49871
2016-04-06T08:19:34.103+05:30 [Info] logWriterStat:: 0 FlushedCount 10590000 QueuedCount 49955
2016-04-06T08:19:34.191+05:30 [Info] logWriterStat:: 0 FlushedCount 10600000 QueuedCount 49972
2016-04-06T08:19:34.279+05:30 [Info] logWriterStat:: 0 FlushedCount 10610000 QueuedCount 50000
2016-04-06T08:19:34.366+05:30 [Info] logWriterStat:: 0 FlushedCount 10620000 QueuedCount 49978
2016-04-06T08:19:34.463+05:30 [Info] logWriterStat:: 0 FlushedCount 10630000 QueuedCount 49975
2016-04-06T08:19:34.546+05:30 [Info] logWriterStat:: 0 FlushedCount 10640000 QueuedCount 49957
2016-04-06T08:19:34.631+05:30 [Info] logWriterStat:: 0 FlushedCount 10650000 QueuedCount 49985
2016-04-06T08:19:34.719+05:30 [Info] logWriterStat:: 0 FlushedCount 10660000 QueuedCount 49941
2016-04-06T08:19:34.813+05:30 [Info] logWriterStat:: 0 FlushedCount 10670000 QueuedCount 49914
2016-04-06T08:19:34.9+05:30 [Info] logWriterStat:: 0 FlushedCount 10680000 QueuedCount 49927
2016-04-06T08:19:34.991+05:30 [Info] logWriterStat:: 0 FlushedCount 10690000 QueuedCount 49938
2016-04-06T08:19:35.078+05:30 [Info] logWriterStat:: 0 FlushedCount 10700000 QueuedCount 49957
2016-04-06T08:19:35.165+05:30 [Info] logWriterStat:: 0 FlushedCount 10710000 QueuedCount 49919
2016-04-06T08:19:35.256+05:30 [Info] logWriterStat:: 0 FlushedCount 10720000 QueuedCount 49993
2016-04-06T08:19:35.344+05:30 [Info] logWriterStat:: 0 FlushedCount 10730000 QueuedCount 49928
2016-04-06T08:19:35.432+05:30 [Info] logWriterStat:: 0 FlushedCount 10740000 QueuedCount 49982
2016-04-06T08:19:35.53+05:30 [Info] logWriterStat:: 0 FlushedCount 10750000 QueuedCount 49473
2016-04-06T08:19:35.619+05:30 [Info] logWriterStat:: 0 FlushedCount 10760000 QueuedCount 49921
2016-04-06T08:19:35.708+05:30 [Info] logWriterStat:: 0 FlushedCount 10770000 QueuedCount 49898
2016-04-06T08:19:35.809+05:30 [Info] logWriterStat:: 0 FlushedCount 10780000 QueuedCount 49924
2016-04-06T08:19:35.9+05:30 [Info] logWriterStat:: 0 FlushedCount 10790000 QueuedCount 49992
2016-04-06T08:19:35.989+05:30 [Info] logWriterStat:: 0 FlushedCount 10800000 QueuedCount 49949
2016-04-06T08:19:36.077+05:30 [Info] logWriterStat:: 0 FlushedCount 10810000 QueuedCount 49992
2016-04-06T08:19:36.164+05:30 [Info] logWriterStat:: 0 FlushedCount 10820000 QueuedCount 49992
2016-04-06T08:19:36.253+05:30 [Info] logWriterStat:: 0 FlushedCount 10830000 QueuedCount 49956
2016-04-06T08:19:36.345+05:30 [Info] logWriterStat:: 0 FlushedCount 10840000 QueuedCount 49978
2016-04-06T08:19:36.425+05:30 [Info] logWriterStat:: 0 FlushedCount 10850000 QueuedCount 43441
2016-04-06T08:19:36.525+05:30 [Info] logWriterStat:: 0 FlushedCount 10860000 QueuedCount 49874
2016-04-06T08:19:36.615+05:30 [Info] logWriterStat:: 0 FlushedCount 10870000 QueuedCount 49968
2016-04-06T08:19:36.706+05:30 [Info] logWriterStat:: 0 FlushedCount 10880000 QueuedCount 49977
2016-04-06T08:19:36.793+05:30 [Info] logWriterStat:: 0 FlushedCount 10890000 QueuedCount 49967
2016-04-06T08:19:36.88+05:30 [Info] logWriterStat:: 0 FlushedCount 10900000 QueuedCount 49988
2016-04-06T08:19:36.97+05:30 [Info] logWriterStat:: 0 FlushedCount 10910000 QueuedCount 49899
2016-04-06T08:19:37.056+05:30 [Info] logWriterStat:: 0 FlushedCount 10920000 QueuedCount 49937
2016-04-06T08:19:37.144+05:30 [Info] logWriterStat:: 0 FlushedCount 10930000 QueuedCount 49961
2016-04-06T08:19:37.233+05:30 [Info] logWriterStat:: 0 FlushedCount 10940000 QueuedCount 49975
2016-04-06T08:19:37.32+05:30 [Info] logWriterStat:: 0 FlushedCount 10950000 QueuedCount 49982
2016-04-06T08:19:37.406+05:30 [Info] logWriterStat:: 0 FlushedCount 10960000 QueuedCount 49543
2016-04-06T08:19:37.518+05:30 [Info] logWriterStat:: 0 FlushedCount 10970000 QueuedCount 49822
2016-04-06T08:19:37.611+05:30 [Info] logWriterStat:: 0 FlushedCount 10980000 QueuedCount 49973
2016-04-06T08:19:37.695+05:30 [Info] logWriterStat:: 0 FlushedCount 10990000 QueuedCount 49934
2016-04-06T08:19:37.784+05:30 [Info] logWriterStat:: 0 FlushedCount 11000000 QueuedCount 49920
2016-04-06T08:19:37.858+05:30 [Info] logWriterStat:: 0 FlushedCount 11010000 QueuedCount 47257
2016-04-06T08:19:37.947+05:30 [Info] logWriterStat:: 0 FlushedCount 11020000 QueuedCount 49918
2016-04-06T08:19:38.033+05:30 [Info] logWriterStat:: 0 FlushedCount 11030000 QueuedCount 49885
2016-04-06T08:19:38.123+05:30 [Info] logWriterStat:: 0 FlushedCount 11040000 QueuedCount 49689
2016-04-06T08:19:38.205+05:30 [Info] logWriterStat:: 0 FlushedCount 11050000 QueuedCount 49967
2016-04-06T08:19:38.292+05:30 [Info] logWriterStat:: 0 FlushedCount 11060000 QueuedCount 49896
2016-04-06T08:19:38.382+05:30 [Info] logWriterStat:: 0 FlushedCount 11070000 QueuedCount 49984
2016-04-06T08:19:38.485+05:30 [Info] logWriterStat:: 0 FlushedCount 11080000 QueuedCount 49965
2016-04-06T08:19:38.574+05:30 [Info] logWriterStat:: 0 FlushedCount 11090000 QueuedCount 49988
2016-04-06T08:19:38.66+05:30 [Info] logWriterStat:: 0 FlushedCount 11100000 QueuedCount 49933
2016-04-06T08:19:38.751+05:30 [Info] logWriterStat:: 0 FlushedCount 11110000 QueuedCount 49978
2016-04-06T08:19:38.84+05:30 [Info] logWriterStat:: 0 FlushedCount 11120000 QueuedCount 49968
2016-04-06T08:19:38.933+05:30 [Info] logWriterStat:: 0 FlushedCount 11130000 QueuedCount 49966
2016-04-06T08:19:39.027+05:30 [Info] logWriterStat:: 0 FlushedCount 11140000 QueuedCount 49887
2016-04-06T08:19:39.129+05:30 [Info] logWriterStat:: 0 FlushedCount 11150000 QueuedCount 49843
2016-04-06T08:19:39.218+05:30 [Info] logWriterStat:: 0 FlushedCount 11160000 QueuedCount 49971
2016-04-06T08:19:39.31+05:30 [Info] logWriterStat:: 0 FlushedCount 11170000 QueuedCount 49930
2016-04-06T08:19:39.395+05:30 [Info] logWriterStat:: 0 FlushedCount 11180000 QueuedCount 49987
2016-04-06T08:19:39.494+05:30 [Info] logWriterStat:: 0 FlushedCount 11190000 QueuedCount 49902
2016-04-06T08:19:39.58+05:30 [Info] logWriterStat:: 0 FlushedCount 11200000 QueuedCount 49961
2016-04-06T08:19:39.658+05:30 [Info] logWriterStat:: 0 FlushedCount 11210000 QueuedCount 49946
2016-04-06T08:19:39.76+05:30 [Info] logWriterStat:: 0 FlushedCount 11220000 QueuedCount 49990
2016-04-06T08:19:39.852+05:30 [Info] logWriterStat:: 0 FlushedCount 11230000 QueuedCount 49931
2016-04-06T08:19:39.938+05:30 [Info] logWriterStat:: 0 FlushedCount 11240000 QueuedCount 49939
2016-04-06T08:19:40.026+05:30 [Info] logWriterStat:: 0 FlushedCount 11250000 QueuedCount 49951
2016-04-06T08:19:40.109+05:30 [Info] logWriterStat:: 0 FlushedCount 11260000 QueuedCount 49483
2016-04-06T08:19:40.194+05:30 [Info] logWriterStat:: 0 FlushedCount 11270000 QueuedCount 49969
2016-04-06T08:19:40.281+05:30 [Info] logWriterStat:: 0 FlushedCount 11280000 QueuedCount 49955
2016-04-06T08:19:40.37+05:30 [Info] logWriterStat:: 0 FlushedCount 11290000 QueuedCount 49894
2016-04-06T08:19:40.484+05:30 [Info] logWriterStat:: 0 FlushedCount 11300000 QueuedCount 49987
2016-04-06T08:19:40.603+05:30 [Info] logWriterStat:: 0 FlushedCount 11310000 QueuedCount 49979
2016-04-06T08:19:40.69+05:30 [Info] logWriterStat:: 0 FlushedCount 11320000 QueuedCount 49946
2016-04-06T08:19:40.787+05:30 [Info] logWriterStat:: 0 FlushedCount 11330000 QueuedCount 49968
2016-04-06T08:19:40.874+05:30 [Info] logWriterStat:: 0 FlushedCount 11340000 QueuedCount 49987
2016-04-06T08:19:40.961+05:30 [Info] logWriterStat:: 0 FlushedCount 11350000 QueuedCount 49980
2016-04-06T08:19:41.045+05:30 [Info] logWriterStat:: 0 FlushedCount 11360000 QueuedCount 49926
2016-04-06T08:19:41.126+05:30 [Info] logWriterStat:: 0 FlushedCount 11370000 QueuedCount 49979
2016-04-06T08:19:41.214+05:30 [Info] logWriterStat:: 0 FlushedCount 11380000 QueuedCount 49975
2016-04-06T08:19:41.305+05:30 [Info] logWriterStat:: 0 FlushedCount 11390000 QueuedCount 49967
2016-04-06T08:19:41.394+05:30 [Info] logWriterStat:: 0 FlushedCount 11400000 QueuedCount 49900
2016-04-06T08:19:41.487+05:30 [Info] logWriterStat:: 0 FlushedCount 11410000 QueuedCount 49989
2016-04-06T08:19:41.577+05:30 [Info] logWriterStat:: 0 FlushedCount 11420000 QueuedCount 49976
2016-04-06T08:19:41.663+05:30 [Info] logWriterStat:: 0 FlushedCount 11430000 QueuedCount 49962
2016-04-06T08:19:41.754+05:30 [Info] logWriterStat:: 0 FlushedCount 11440000 QueuedCount 49989
2016-04-06T08:19:41.841+05:30 [Info] logWriterStat:: 0 FlushedCount 11450000 QueuedCount 49933
2016-04-06T08:19:41.923+05:30 [Info] logWriterStat:: 0 FlushedCount 11460000 QueuedCount 49880
2016-04-06T08:19:42.012+05:30 [Info] logWriterStat:: 0 FlushedCount 11470000 QueuedCount 49976
2016-04-06T08:19:42.1+05:30 [Info] logWriterStat:: 0 FlushedCount 11480000 QueuedCount 49974
2016-04-06T08:19:42.189+05:30 [Info] logWriterStat:: 0 FlushedCount 11490000 QueuedCount 49868
2016-04-06T08:19:42.276+05:30 [Info] logWriterStat:: 0 FlushedCount 11500000 QueuedCount 49928
2016-04-06T08:19:42.362+05:30 [Info] logWriterStat:: 0 FlushedCount 11510000 QueuedCount 49893
2016-04-06T08:19:42.458+05:30 [Info] logWriterStat:: 0 FlushedCount 11520000 QueuedCount 49926
2016-04-06T08:19:42.561+05:30 [Info] logWriterStat:: 0 FlushedCount 11530000 QueuedCount 49923
2016-04-06T08:19:42.642+05:30 [Info] logWriterStat:: 0 FlushedCount 11540000 QueuedCount 48815
2016-04-06T08:19:42.716+05:30 [Info] logWriterStat:: 0 FlushedCount 11550000 QueuedCount 49820
2016-04-06T08:19:42.786+05:30 [Info] logWriterStat:: 0 FlushedCount 11560000 QueuedCount 47691
2016-04-06T08:19:42.857+05:30 [Info] logWriterStat:: 0 FlushedCount 11570000 QueuedCount 47715
2016-04-06T08:19:42.934+05:30 [Info] logWriterStat:: 0 FlushedCount 11580000 QueuedCount 49034
2016-04-06T08:19:43.012+05:30 [Info] logWriterStat:: 0 FlushedCount 11590000 QueuedCount 43566
2016-04-06T08:19:43.092+05:30 [Info] logWriterStat:: 0 FlushedCount 11600000 QueuedCount 46606
2016-04-06T08:19:43.165+05:30 [Info] logWriterStat:: 0 FlushedCount 11610000 QueuedCount 48957
2016-04-06T08:19:43.234+05:30 [Info] logWriterStat:: 0 FlushedCount 11620000 QueuedCount 46670
2016-04-06T08:19:43.31+05:30 [Info] logWriterStat:: 0 FlushedCount 11630000 QueuedCount 44426
2016-04-06T08:19:43.387+05:30 [Info] logWriterStat:: 0 FlushedCount 11640000 QueuedCount 45676
2016-04-06T08:19:43.478+05:30 [Info] logWriterStat:: 0 FlushedCount 11650000 QueuedCount 47871
2016-04-06T08:19:43.552+05:30 [Info] logWriterStat:: 0 FlushedCount 11660000 QueuedCount 45382
2016-04-06T08:19:43.628+05:30 [Info] logWriterStat:: 0 FlushedCount 11670000 QueuedCount 45947
2016-04-06T08:19:43.708+05:30 [Info] logWriterStat:: 0 FlushedCount 11680000 QueuedCount 47128
2016-04-06T08:19:43.783+05:30 [Info] logWriterStat:: 0 FlushedCount 11690000 QueuedCount 47862
2016-04-06T08:19:43.865+05:30 [Info] logWriterStat:: 0 FlushedCount 11700000 QueuedCount 48246
2016-04-06T08:19:43.946+05:30 [Info] logWriterStat:: 0 FlushedCount 11710000 QueuedCount 49961
2016-04-06T08:19:44.027+05:30 [Info] logWriterStat:: 0 FlushedCount 11720000 QueuedCount 47857
2016-04-06T08:19:44.115+05:30 [Info] logWriterStat:: 0 FlushedCount 11730000 QueuedCount 48795
2016-04-06T08:19:44.19+05:30 [Info] logWriterStat:: 0 FlushedCount 11740000 QueuedCount 46010
2016-04-06T08:19:44.274+05:30 [Info] logWriterStat:: 0 FlushedCount 11750000 QueuedCount 44902
2016-04-06T08:19:44.348+05:30 [Info] logWriterStat:: 0 FlushedCount 11760000 QueuedCount 47911
2016-04-06T08:19:44.422+05:30 [Info] logWriterStat:: 0 FlushedCount 11770000 QueuedCount 45728
2016-04-06T08:19:44.514+05:30 [Info] logWriterStat:: 0 FlushedCount 11780000 QueuedCount 43841
2016-04-06T08:19:44.591+05:30 [Info] logWriterStat:: 0 FlushedCount 11790000 QueuedCount 45563
2016-04-06T08:19:44.671+05:30 [Info] logWriterStat:: 0 FlushedCount 11800000 QueuedCount 45712
2016-04-06T08:19:44.748+05:30 [Info] logWriterStat:: 0 FlushedCount 11810000 QueuedCount 46561
2016-04-06T08:19:44.825+05:30 [Info] logWriterStat:: 0 FlushedCount 11820000 QueuedCount 43515
2016-04-06T08:19:44.897+05:30 [Info] logWriterStat:: 0 FlushedCount 11830000 QueuedCount 47359
2016-04-06T08:19:44.987+05:30 [Info] logWriterStat:: 0 FlushedCount 11840000 QueuedCount 47875
2016-04-06T08:19:45.084+05:30 [Info] logWriterStat:: 0 FlushedCount 11850000 QueuedCount 49006
2016-04-06T08:19:45.182+05:30 [Info] logWriterStat:: 0 FlushedCount 11860000 QueuedCount 46815
2016-04-06T08:19:45.281+05:30 [Info] logWriterStat:: 0 FlushedCount 11870000 QueuedCount 48071
2016-04-06T08:19:45.378+05:30 [Info] logWriterStat:: 0 FlushedCount 11880000 QueuedCount 48309
2016-04-06T08:19:45.489+05:30 [Info] logWriterStat:: 0 FlushedCount 11890000 QueuedCount 47406
2016-04-06T08:19:45.597+05:30 [Info] logWriterStat:: 0 FlushedCount 11900000 QueuedCount 47378
2016-04-06T08:19:45.702+05:30 [Info] logWriterStat:: 0 FlushedCount 11910000 QueuedCount 48244
2016-04-06T08:19:45.815+05:30 [Info] logWriterStat:: 0 FlushedCount 11920000 QueuedCount 49864
2016-04-06T08:19:45.906+05:30 [Info] logWriterStat:: 0 FlushedCount 11930000 QueuedCount 49896
2016-04-06T08:19:46.008+05:30 [Info] logWriterStat:: 0 FlushedCount 11940000 QueuedCount 49984
2016-04-06T08:19:46.105+05:30 [Info] logWriterStat:: 0 FlushedCount 11950000 QueuedCount 49615
2016-04-06T08:19:46.193+05:30 [Info] logWriterStat:: 0 FlushedCount 11960000 QueuedCount 49576
2016-04-06T08:19:46.291+05:30 [Info] logWriterStat:: 0 FlushedCount 11970000 QueuedCount 49943
2016-04-06T08:19:46.384+05:30 [Info] logWriterStat:: 0 FlushedCount 11980000 QueuedCount 49923
2016-04-06T08:19:46.494+05:30 [Info] logWriterStat:: 0 FlushedCount 11990000 QueuedCount 49863
2016-04-06T08:19:46.591+05:30 [Info] logWriterStat:: 0 FlushedCount 12000000 QueuedCount 49825
2016-04-06T08:19:46.686+05:30 [Info] logWriterStat:: 0 FlushedCount 12010000 QueuedCount 49938
2016-04-06T08:19:46.783+05:30 [Info] logWriterStat:: 0 FlushedCount 12020000 QueuedCount 49851
2016-04-06T08:19:46.879+05:30 [Info] logWriterStat:: 0 FlushedCount 12030000 QueuedCount 49928
2016-04-06T08:19:46.975+05:30 [Info] logWriterStat:: 0 FlushedCount 12040000 QueuedCount 48857
2016-04-06T08:19:47.065+05:30 [Info] logWriterStat:: 0 FlushedCount 12050000 QueuedCount 48703
2016-04-06T08:19:47.159+05:30 [Info] logWriterStat:: 0 FlushedCount 12060000 QueuedCount 47340
2016-04-06T08:19:47.253+05:30 [Info] logWriterStat:: 0 FlushedCount 12070000 QueuedCount 47473
2016-04-06T08:19:47.35+05:30 [Info] logWriterStat:: 0 FlushedCount 12080000 QueuedCount 48130
2016-04-06T08:19:47.458+05:30 [Info] logWriterStat:: 0 FlushedCount 12090000 QueuedCount 49986
2016-04-06T08:19:47.574+05:30 [Info] logWriterStat:: 0 FlushedCount 12100000 QueuedCount 49903
2016-04-06T08:19:47.674+05:30 [Info] logWriterStat:: 0 FlushedCount 12110000 QueuedCount 49954
2016-04-06T08:19:47.771+05:30 [Info] logWriterStat:: 0 FlushedCount 12120000 QueuedCount 49926
2016-04-06T08:19:47.876+05:30 [Info] logWriterStat:: 0 FlushedCount 12130000 QueuedCount 49976
2016-04-06T08:19:47.969+05:30 [Info] logWriterStat:: 0 FlushedCount 12140000 QueuedCount 48811
2016-04-06T08:19:48.06+05:30 [Info] logWriterStat:: 0 FlushedCount 12150000 QueuedCount 49929
2016-04-06T08:19:48.153+05:30 [Info] logWriterStat:: 0 FlushedCount 12160000 QueuedCount 49998
2016-04-06T08:19:48.25+05:30 [Info] logWriterStat:: 0 FlushedCount 12170000 QueuedCount 47874
2016-04-06T08:19:48.341+05:30 [Info] logWriterStat:: 0 FlushedCount 12180000 QueuedCount 47116
2016-04-06T08:19:48.444+05:30 [Info] logWriterStat:: 0 FlushedCount 12190000 QueuedCount 47630
2016-04-06T08:19:48.54+05:30 [Info] logWriterStat:: 0 FlushedCount 12200000 QueuedCount 47353
2016-04-06T08:19:48.632+05:30 [Info] logWriterStat:: 0 FlushedCount 12210000 QueuedCount 47602
2016-04-06T08:19:48.734+05:30 [Info] logWriterStat:: 0 FlushedCount 12220000 QueuedCount 46779
2016-04-06T08:19:48.828+05:30 [Info] logWriterStat:: 0 FlushedCount 12230000 QueuedCount 47199
2016-04-06T08:19:48.917+05:30 [Info] logWriterStat:: 0 FlushedCount 12240000 QueuedCount 46999
2016-04-06T08:19:49.013+05:30 [Info] logWriterStat:: 0 FlushedCount 12250000 QueuedCount 47900
2016-04-06T08:19:49.112+05:30 [Info] logWriterStat:: 0 FlushedCount 12260000 QueuedCount 48540
2016-04-06T08:19:49.207+05:30 [Info] logWriterStat:: 0 FlushedCount 12270000 QueuedCount 47751
2016-04-06T08:19:49.301+05:30 [Info] logWriterStat:: 0 FlushedCount 12280000 QueuedCount 47946
2016-04-06T08:19:49.399+05:30 [Info] logWriterStat:: 0 FlushedCount 12290000 QueuedCount 47380
2016-04-06T08:19:49.504+05:30 [Info] logWriterStat:: 0 FlushedCount 12300000 QueuedCount 47658
2016-04-06T08:19:49.61+05:30 [Info] logWriterStat:: 0 FlushedCount 12310000 QueuedCount 48507
2016-04-06T08:19:49.692+05:30 [Info] logWriterStat:: 0 FlushedCount 12320000 QueuedCount 46579
2016-04-06T08:19:49.78+05:30 [Info] logWriterStat:: 0 FlushedCount 12330000 QueuedCount 48297
2016-04-06T08:19:49.872+05:30 [Info] logWriterStat:: 0 FlushedCount 12340000 QueuedCount 48082
2016-04-06T08:19:49.963+05:30 [Info] logWriterStat:: 0 FlushedCount 12350000 QueuedCount 48558
2016-04-06T08:19:50.054+05:30 [Info] logWriterStat:: 0 FlushedCount 12360000 QueuedCount 48410
2016-04-06T08:19:50.15+05:30 [Info] logWriterStat:: 0 FlushedCount 12370000 QueuedCount 47973
2016-04-06T08:19:50.245+05:30 [Info] logWriterStat:: 0 FlushedCount 12380000 QueuedCount 48358
2016-04-06T08:19:50.343+05:30 [Info] logWriterStat:: 0 FlushedCount 12390000 QueuedCount 47849
2016-04-06T08:19:50.435+05:30 [Info] logWriterStat:: 0 FlushedCount 12400000 QueuedCount 47917
2016-04-06T08:19:50.584+05:30 [Info] logWriterStat:: 0 FlushedCount 12410000 QueuedCount 49988
2016-04-06T08:19:50.675+05:30 [Info] logWriterStat:: 0 FlushedCount 12420000 QueuedCount 49922
2016-04-06T08:19:50.785+05:30 [Info] logWriterStat:: 0 FlushedCount 12430000 QueuedCount 49979
2016-04-06T08:19:50.877+05:30 [Info] logWriterStat:: 0 FlushedCount 12440000 QueuedCount 49968
2016-04-06T08:19:50.979+05:30 [Info] logWriterStat:: 0 FlushedCount 12450000 QueuedCount 49911
2016-04-06T08:19:51.072+05:30 [Info] logWriterStat:: 0 FlushedCount 12460000 QueuedCount 49872
2016-04-06T08:19:51.164+05:30 [Info] logWriterStat:: 0 FlushedCount 12470000 QueuedCount 49866
2016-04-06T08:19:51.255+05:30 [Info] logWriterStat:: 0 FlushedCount 12480000 QueuedCount 49981
2016-04-06T08:19:51.352+05:30 [Info] logWriterStat:: 0 FlushedCount 12490000 QueuedCount 49958
2016-04-06T08:19:51.456+05:30 [Info] logWriterStat:: 0 FlushedCount 12500000 QueuedCount 49968
2016-04-06T08:19:51.664+05:30 [Info] logWriterStat:: 0 FlushedCount 12510000 QueuedCount 48636
2016-04-06T08:19:51.918+05:30 [Info] logWriterStat:: 0 FlushedCount 12520000 QueuedCount 49851
2016-04-06T08:19:52.094+05:30 [Info] logWriterStat:: 0 FlushedCount 12530000 QueuedCount 49840
2016-04-06T08:19:52.272+05:30 [Info] logWriterStat:: 0 FlushedCount 12540000 QueuedCount 48690
2016-04-06T08:19:52.518+05:30 [Info] logWriterStat:: 0 FlushedCount 12550000 QueuedCount 49793
2016-04-06T08:19:52.723+05:30 [Info] logWriterStat:: 0 FlushedCount 12560000 QueuedCount 49762
2016-04-06T08:19:52.925+05:30 [Info] logWriterStat:: 0 FlushedCount 12570000 QueuedCount 48812
2016-04-06T08:19:53.134+05:30 [Info] logWriterStat:: 0 FlushedCount 12580000 QueuedCount 48609
2016-04-06T08:19:53.347+05:30 [Info] logWriterStat:: 0 FlushedCount 12590000 QueuedCount 49885
2016-04-06T08:19:53.501+05:30 [Info] logWriterStat:: 0 FlushedCount 12600000 QueuedCount 49992
2016-04-06T08:19:53.635+05:30 [Info] logWriterStat:: 0 FlushedCount 12610000 QueuedCount 49941
2016-04-06T08:19:53.758+05:30 [Info] logWriterStat:: 0 FlushedCount 12620000 QueuedCount 49952
2016-04-06T08:19:53.866+05:30 [Info] logWriterStat:: 0 FlushedCount 12630000 QueuedCount 48168
2016-04-06T08:19:53.959+05:30 [Info] logWriterStat:: 0 FlushedCount 12640000 QueuedCount 49977
2016-04-06T08:19:54.089+05:30 [Info] logWriterStat:: 0 FlushedCount 12650000 QueuedCount 49866
2016-04-06T08:19:54.22+05:30 [Info] logWriterStat:: 0 FlushedCount 12660000 QueuedCount 49957
2016-04-06T08:19:54.324+05:30 [Info] logWriterStat:: 0 FlushedCount 12670000 QueuedCount 49975
2016-04-06T08:19:54.456+05:30 [Info] logWriterStat:: 0 FlushedCount 12680000 QueuedCount 49930
2016-04-06T08:19:54.595+05:30 [Info] logWriterStat:: 0 FlushedCount 12690000 QueuedCount 49896
2016-04-06T08:19:54.743+05:30 [Info] logWriterStat:: 0 FlushedCount 12700000 QueuedCount 49926
2016-04-06T08:19:54.851+05:30 [Info] logWriterStat:: 0 FlushedCount 12710000 QueuedCount 49869
2016-04-06T08:19:54.983+05:30 [Info] logWriterStat:: 0 FlushedCount 12720000 QueuedCount 49964
2016-04-06T08:19:55.106+05:30 [Info] logWriterStat:: 0 FlushedCount 12730000 QueuedCount 49956
2016-04-06T08:19:55.234+05:30 [Info] logWriterStat:: 0 FlushedCount 12740000 QueuedCount 49935
2016-04-06T08:19:55.318+05:30 [Info] logWriterStat:: 0 FlushedCount 12750000 QueuedCount 49963
2016-04-06T08:19:55.4+05:30 [Info] logWriterStat:: 0 FlushedCount 12760000 QueuedCount 49967
2016-04-06T08:19:55.485+05:30 [Info] logWriterStat:: 0 FlushedCount 12770000 QueuedCount 49953
2016-04-06T08:19:55.566+05:30 [Info] logWriterStat:: 0 FlushedCount 12780000 QueuedCount 49975
2016-04-06T08:19:55.651+05:30 [Info] logWriterStat:: 0 FlushedCount 12790000 QueuedCount 49915
2016-04-06T08:19:55.733+05:30 [Info] logWriterStat:: 0 FlushedCount 12800000 QueuedCount 49992
2016-04-06T08:19:55.825+05:30 [Info] logWriterStat:: 0 FlushedCount 12810000 QueuedCount 49393
2016-04-06T08:19:55.898+05:30 [Info] logWriterStat:: 0 FlushedCount 12820000 QueuedCount 49980
2016-04-06T08:19:55.977+05:30 [Info] logWriterStat:: 0 FlushedCount 12830000 QueuedCount 49941
2016-04-06T08:19:56.055+05:30 [Info] logWriterStat:: 0 FlushedCount 12840000 QueuedCount 49999
2016-04-06T08:19:56.14+05:30 [Info] logWriterStat:: 0 FlushedCount 12850000 QueuedCount 49966
2016-04-06T08:19:56.218+05:30 [Info] logWriterStat:: 0 FlushedCount 12860000 QueuedCount 49942
2016-04-06T08:19:56.299+05:30 [Info] logWriterStat:: 0 FlushedCount 12870000 QueuedCount 49984
2016-04-06T08:19:56.379+05:30 [Info] logWriterStat:: 0 FlushedCount 12880000 QueuedCount 49892
2016-04-06T08:19:56.465+05:30 [Info] logWriterStat:: 0 FlushedCount 12890000 QueuedCount 49318
2016-04-06T08:19:56.545+05:30 [Info] logWriterStat:: 0 FlushedCount 12900000 QueuedCount 49938
2016-04-06T08:19:56.629+05:30 [Info] logWriterStat:: 0 FlushedCount 12910000 QueuedCount 49986
2016-04-06T08:19:56.71+05:30 [Info] logWriterStat:: 0 FlushedCount 12920000 QueuedCount 49956
2016-04-06T08:19:56.79+05:30 [Info] logWriterStat:: 0 FlushedCount 12930000 QueuedCount 49942
2016-04-06T08:19:56.87+05:30 [Info] logWriterStat:: 0 FlushedCount 12940000 QueuedCount 49937
2016-04-06T08:19:56.951+05:30 [Info] logWriterStat:: 0 FlushedCount 12950000 QueuedCount 49983
2016-04-06T08:19:57.031+05:30 [Info] logWriterStat:: 0 FlushedCount 12960000 QueuedCount 49967
2016-04-06T08:19:57.11+05:30 [Info] logWriterStat:: 0 FlushedCount 12970000 QueuedCount 49919
2016-04-06T08:19:57.183+05:30 [Info] logWriterStat:: 0 FlushedCount 12980000 QueuedCount 49990
2016-04-06T08:19:57.261+05:30 [Info] logWriterStat:: 0 FlushedCount 12990000 QueuedCount 49811
2016-04-06T08:19:57.344+05:30 [Info] logWriterStat:: 0 FlushedCount 13000000 QueuedCount 49989
2016-04-06T08:19:57.423+05:30 [Info] logWriterStat:: 0 FlushedCount 13010000 QueuedCount 49835
2016-04-06T08:19:57.509+05:30 [Info] logWriterStat:: 0 FlushedCount 13020000 QueuedCount 48595
2016-04-06T08:19:57.592+05:30 [Info] logWriterStat:: 0 FlushedCount 13030000 QueuedCount 49936
2016-04-06T08:19:57.691+05:30 [Info] logWriterStat:: 0 FlushedCount 13040000 QueuedCount 49937
2016-04-06T08:19:57.778+05:30 [Info] logWriterStat:: 0 FlushedCount 13050000 QueuedCount 49931
2016-04-06T08:19:57.861+05:30 [Info] logWriterStat:: 0 FlushedCount 13060000 QueuedCount 49877
2016-04-06T08:19:57.942+05:30 [Info] logWriterStat:: 0 FlushedCount 13070000 QueuedCount 49968
2016-04-06T08:19:58.025+05:30 [Info] logWriterStat:: 0 FlushedCount 13080000 QueuedCount 49979
2016-04-06T08:19:58.102+05:30 [Info] logWriterStat:: 0 FlushedCount 13090000 QueuedCount 49884
2016-04-06T08:19:58.175+05:30 [Info] logWriterStat:: 0 FlushedCount 13100000 QueuedCount 45434
2016-04-06T08:19:58.248+05:30 [Info] logWriterStat:: 0 FlushedCount 13110000 QueuedCount 42833
2016-04-06T08:19:58.322+05:30 [Info] logWriterStat:: 0 FlushedCount 13120000 QueuedCount 49943
2016-04-06T08:19:58.401+05:30 [Info] logWriterStat:: 0 FlushedCount 13130000 QueuedCount 49928
2016-04-06T08:19:58.492+05:30 [Info] logWriterStat:: 0 FlushedCount 13140000 QueuedCount 49994
2016-04-06T08:19:58.573+05:30 [Info] logWriterStat:: 0 FlushedCount 13150000 QueuedCount 49951
2016-04-06T08:19:58.649+05:30 [Info] logWriterStat:: 0 FlushedCount 13160000 QueuedCount 49171
2016-04-06T08:19:58.73+05:30 [Info] logWriterStat:: 0 FlushedCount 13170000 QueuedCount 49988
2016-04-06T08:19:58.811+05:30 [Info] logWriterStat:: 0 FlushedCount 13180000 QueuedCount 49969
2016-04-06T08:19:58.893+05:30 [Info] logWriterStat:: 0 FlushedCount 13190000 QueuedCount 49982
2016-04-06T08:19:58.974+05:30 [Info] logWriterStat:: 0 FlushedCount 13200000 QueuedCount 49974
2016-04-06T08:19:59.062+05:30 [Info] logWriterStat:: 0 FlushedCount 13210000 QueuedCount 49993
2016-04-06T08:19:59.145+05:30 [Info] logWriterStat:: 0 FlushedCount 13220000 QueuedCount 49948
2016-04-06T08:19:59.225+05:30 [Info] logWriterStat:: 0 FlushedCount 13230000 QueuedCount 49964
2016-04-06T08:19:59.307+05:30 [Info] logWriterStat:: 0 FlushedCount 13240000 QueuedCount 49973
2016-04-06T08:19:59.383+05:30 [Info] logWriterStat:: 0 FlushedCount 13250000 QueuedCount 49947
2016-04-06T08:19:59.476+05:30 [Info] logWriterStat:: 0 FlushedCount 13260000 QueuedCount 49976
2016-04-06T08:19:59.556+05:30 [Info] logWriterStat:: 0 FlushedCount 13270000 QueuedCount 49915
2016-04-06T08:19:59.637+05:30 [Info] logWriterStat:: 0 FlushedCount 13280000 QueuedCount 49969
2016-04-06T08:19:59.717+05:30 [Info] logWriterStat:: 0 FlushedCount 13290000 QueuedCount 49920
2016-04-06T08:19:59.798+05:30 [Info] logWriterStat:: 0 FlushedCount 13300000 QueuedCount 49995
2016-04-06T08:19:59.878+05:30 [Info] logWriterStat:: 0 FlushedCount 13310000 QueuedCount 49959
2016-04-06T08:19:59.959+05:30 [Info] logWriterStat:: 0 FlushedCount 13320000 QueuedCount 49961
2016-04-06T08:20:00.039+05:30 [Info] logWriterStat:: 0 FlushedCount 13330000 QueuedCount 49972
2016-04-06T08:20:00.121+05:30 [Info] logWriterStat:: 0 FlushedCount 13340000 QueuedCount 49987
2016-04-06T08:20:00.198+05:30 [Info] logWriterStat:: 0 FlushedCount 13350000 QueuedCount 49972
2016-04-06T08:20:00.279+05:30 [Info] logWriterStat:: 0 FlushedCount 13360000 QueuedCount 49988
2016-04-06T08:20:00.358+05:30 [Info] logWriterStat:: 0 FlushedCount 13370000 QueuedCount 49860
2016-04-06T08:20:00.46+05:30 [Info] logWriterStat:: 0 FlushedCount 13380000 QueuedCount 49996
2016-04-06T08:20:00.578+05:30 [Info] logWriterStat:: 0 FlushedCount 13390000 QueuedCount 49961
2016-04-06T08:20:00.66+05:30 [Info] logWriterStat:: 0 FlushedCount 13400000 QueuedCount 49972
2016-04-06T08:20:00.745+05:30 [Info] logWriterStat:: 0 FlushedCount 13410000 QueuedCount 49514
2016-04-06T08:20:00.835+05:30 [Info] logWriterStat:: 0 FlushedCount 13420000 QueuedCount 49977
2016-04-06T08:20:00.919+05:30 [Info] logWriterStat:: 0 FlushedCount 13430000 QueuedCount 49937
2016-04-06T08:20:01.003+05:30 [Info] logWriterStat:: 0 FlushedCount 13440000 QueuedCount 49967
2016-04-06T08:20:01.089+05:30 [Info] logWriterStat:: 0 FlushedCount 13450000 QueuedCount 49948
2016-04-06T08:20:01.17+05:30 [Info] logWriterStat:: 0 FlushedCount 13460000 QueuedCount 49968
2016-04-06T08:20:01.248+05:30 [Info] logWriterStat:: 0 FlushedCount 13470000 QueuedCount 49996
2016-04-06T08:20:01.332+05:30 [Info] logWriterStat:: 0 FlushedCount 13480000 QueuedCount 49883
2016-04-06T08:20:01.415+05:30 [Info] logWriterStat:: 0 FlushedCount 13490000 QueuedCount 49975
2016-04-06T08:20:01.507+05:30 [Info] logWriterStat:: 0 FlushedCount 13500000 QueuedCount 49965
2016-04-06T08:20:01.59+05:30 [Info] logWriterStat:: 0 FlushedCount 13510000 QueuedCount 49977
2016-04-06T08:20:01.671+05:30 [Info] logWriterStat:: 0 FlushedCount 13520000 QueuedCount 49972
2016-04-06T08:20:01.752+05:30 [Info] logWriterStat:: 0 FlushedCount 13530000 QueuedCount 49949
2016-04-06T08:20:01.834+05:30 [Info] logWriterStat:: 0 FlushedCount 13540000 QueuedCount 49975
2016-04-06T08:20:01.916+05:30 [Info] logWriterStat:: 0 FlushedCount 13550000 QueuedCount 49992
2016-04-06T08:20:02.002+05:30 [Info] logWriterStat:: 0 FlushedCount 13560000 QueuedCount 49927
2016-04-06T08:20:02.083+05:30 [Info] logWriterStat:: 0 FlushedCount 13570000 QueuedCount 49998
2016-04-06T08:20:02.167+05:30 [Info] logWriterStat:: 0 FlushedCount 13580000 QueuedCount 49929
2016-04-06T08:20:02.249+05:30 [Info] logWriterStat:: 0 FlushedCount 13590000 QueuedCount 49935
2016-04-06T08:20:02.331+05:30 [Info] logWriterStat:: 0 FlushedCount 13600000 QueuedCount 49957
2016-04-06T08:20:02.412+05:30 [Info] logWriterStat:: 0 FlushedCount 13610000 QueuedCount 49955
2016-04-06T08:20:02.514+05:30 [Info] logWriterStat:: 0 FlushedCount 13620000 QueuedCount 49991
2016-04-06T08:20:02.622+05:30 [Info] logWriterStat:: 0 FlushedCount 13630000 QueuedCount 49973
2016-04-06T08:20:02.737+05:30 [Info] logWriterStat:: 0 FlushedCount 13640000 QueuedCount 49985
2016-04-06T08:20:02.821+05:30 [Info] logWriterStat:: 0 FlushedCount 13650000 QueuedCount 49946
2016-04-06T08:20:02.901+05:30 [Info] logWriterStat:: 0 FlushedCount 13660000 QueuedCount 49974
2016-04-06T08:20:02.981+05:30 [Info] logWriterStat:: 0 FlushedCount 13670000 QueuedCount 49933
2016-04-06T08:20:03.063+05:30 [Info] logWriterStat:: 0 FlushedCount 13680000 QueuedCount 49988
2016-04-06T08:20:03.143+05:30 [Info] logWriterStat:: 0 FlushedCount 13690000 QueuedCount 49972
2016-04-06T08:20:03.225+05:30 [Info] logWriterStat:: 0 FlushedCount 13700000 QueuedCount 49981
2016-04-06T08:20:03.307+05:30 [Info] logWriterStat:: 0 FlushedCount 13710000 QueuedCount 49919
2016-04-06T08:20:03.388+05:30 [Info] logWriterStat:: 0 FlushedCount 13720000 QueuedCount 49972
2016-04-06T08:20:03.478+05:30 [Info] logWriterStat:: 0 FlushedCount 13730000 QueuedCount 49935
2016-04-06T08:20:03.556+05:30 [Info] logWriterStat:: 0 FlushedCount 13740000 QueuedCount 49872
2016-04-06T08:20:03.642+05:30 [Info] logWriterStat:: 0 FlushedCount 13750000 QueuedCount 49911
2016-04-06T08:20:03.737+05:30 [Info] logWriterStat:: 0 FlushedCount 13760000 QueuedCount 49994
2016-04-06T08:20:03.829+05:30 [Info] logWriterStat:: 0 FlushedCount 13770000 QueuedCount 49921
2016-04-06T08:20:03.912+05:30 [Info] logWriterStat:: 0 FlushedCount 13780000 QueuedCount 49982
2016-04-06T08:20:04.002+05:30 [Info] logWriterStat:: 0 FlushedCount 13790000 QueuedCount 49960
2016-04-06T08:20:04.094+05:30 [Info] logWriterStat:: 0 FlushedCount 13800000 QueuedCount 49839
2016-04-06T08:20:04.175+05:30 [Info] logWriterStat:: 0 FlushedCount 13810000 QueuedCount 49935
2016-04-06T08:20:04.258+05:30 [Info] logWriterStat:: 0 FlushedCount 13820000 QueuedCount 49953
2016-04-06T08:20:04.341+05:30 [Info] logWriterStat:: 0 FlushedCount 13830000 QueuedCount 49955
2016-04-06T08:20:04.425+05:30 [Info] logWriterStat:: 0 FlushedCount 13840000 QueuedCount 49976
2016-04-06T08:20:04.52+05:30 [Info] logWriterStat:: 0 FlushedCount 13850000 QueuedCount 49983
2016-04-06T08:20:04.603+05:30 [Info] logWriterStat:: 0 FlushedCount 13860000 QueuedCount 49718
2016-04-06T08:20:04.682+05:30 [Info] logWriterStat:: 0 FlushedCount 13870000 QueuedCount 49944
2016-04-06T08:20:04.763+05:30 [Info] logWriterStat:: 0 FlushedCount 13880000 QueuedCount 49932
2016-04-06T08:20:04.848+05:30 [Info] logWriterStat:: 0 FlushedCount 13890000 QueuedCount 49959
2016-04-06T08:20:04.928+05:30 [Info] logWriterStat:: 0 FlushedCount 13900000 QueuedCount 49882
2016-04-06T08:20:05.01+05:30 [Info] logWriterStat:: 0 FlushedCount 13910000 QueuedCount 49916
2016-04-06T08:20:05.091+05:30 [Info] logWriterStat:: 0 FlushedCount 13920000 QueuedCount 49988
2016-04-06T08:20:05.172+05:30 [Info] logWriterStat:: 0 FlushedCount 13930000 QueuedCount 49832
2016-04-06T08:20:05.249+05:30 [Info] logWriterStat:: 0 FlushedCount 13940000 QueuedCount 49959
2016-04-06T08:20:05.334+05:30 [Info] logWriterStat:: 0 FlushedCount 13950000 QueuedCount 49935
2016-04-06T08:20:05.415+05:30 [Info] logWriterStat:: 0 FlushedCount 13960000 QueuedCount 49976
2016-04-06T08:20:05.523+05:30 [Info] logWriterStat:: 0 FlushedCount 13970000 QueuedCount 49960
2016-04-06T08:20:05.609+05:30 [Info] logWriterStat:: 0 FlushedCount 13980000 QueuedCount 49981
2016-04-06T08:20:05.689+05:30 [Info] logWriterStat:: 0 FlushedCount 13990000 QueuedCount 49911
2016-04-06T08:20:05.778+05:30 [Info] logWriterStat:: 0 FlushedCount 14000000 QueuedCount 49994
2016-04-06T08:20:05.86+05:30 [Info] logWriterStat:: 0 FlushedCount 14010000 QueuedCount 49935
2016-04-06T08:20:05.94+05:30 [Info] logWriterStat:: 0 FlushedCount 14020000 QueuedCount 49958
2016-04-06T08:20:06.024+05:30 [Info] logWriterStat:: 0 FlushedCount 14030000 QueuedCount 49940
2016-04-06T08:20:06.101+05:30 [Info] logWriterStat:: 0 FlushedCount 14040000 QueuedCount 49119
2016-04-06T08:20:06.18+05:30 [Info] logWriterStat:: 0 FlushedCount 14050000 QueuedCount 49978
2016-04-06T08:20:06.26+05:30 [Info] logWriterStat:: 0 FlushedCount 14060000 QueuedCount 49977
2016-04-06T08:20:06.355+05:30 [Info] logWriterStat:: 0 FlushedCount 14070000 QueuedCount 49997
2016-04-06T08:20:06.473+05:30 [Info] logWriterStat:: 0 FlushedCount 14080000 QueuedCount 49991
2016-04-06T08:20:06.574+05:30 [Info] logWriterStat:: 0 FlushedCount 14090000 QueuedCount 49951
2016-04-06T08:20:06.66+05:30 [Info] logWriterStat:: 0 FlushedCount 14100000 QueuedCount 49932
2016-04-06T08:20:06.738+05:30 [Info] logWriterStat:: 0 FlushedCount 14110000 QueuedCount 49874
2016-04-06T08:20:06.817+05:30 [Info] logWriterStat:: 0 FlushedCount 14120000 QueuedCount 49983
2016-04-06T08:20:06.899+05:30 [Info] logWriterStat:: 0 FlushedCount 14130000 QueuedCount 49940
2016-04-06T08:20:06.979+05:30 [Info] logWriterStat:: 0 FlushedCount 14140000 QueuedCount 49986
2016-04-06T08:20:07.06+05:30 [Info] logWriterStat:: 0 FlushedCount 14150000 QueuedCount 49869
2016-04-06T08:20:07.138+05:30 [Info] logWriterStat:: 0 FlushedCount 14160000 QueuedCount 49974
2016-04-06T08:20:07.211+05:30 [Info] logWriterStat:: 0 FlushedCount 14170000 QueuedCount 49976
2016-04-06T08:20:07.293+05:30 [Info] logWriterStat:: 0 FlushedCount 14180000 QueuedCount 49975
2016-04-06T08:20:07.376+05:30 [Info] logWriterStat:: 0 FlushedCount 14190000 QueuedCount 49980
2016-04-06T08:20:07.468+05:30 [Info] logWriterStat:: 0 FlushedCount 14200000 QueuedCount 49987
2016-04-06T08:20:07.551+05:30 [Info] logWriterStat:: 0 FlushedCount 14210000 QueuedCount 49814
2016-04-06T08:20:07.636+05:30 [Info] logWriterStat:: 0 FlushedCount 14220000 QueuedCount 49936
2016-04-06T08:20:07.732+05:30 [Info] logWriterStat:: 0 FlushedCount 14230000 QueuedCount 49983
2016-04-06T08:20:07.818+05:30 [Info] logWriterStat:: 0 FlushedCount 14240000 QueuedCount 49988
2016-04-06T08:20:07.895+05:30 [Info] logWriterStat:: 0 FlushedCount 14250000 QueuedCount 49883
2016-04-06T08:20:07.978+05:30 [Info] logWriterStat:: 0 FlushedCount 14260000 QueuedCount 49989
2016-04-06T08:20:08.06+05:30 [Info] logWriterStat:: 0 FlushedCount 14270000 QueuedCount 49988
2016-04-06T08:20:08.145+05:30 [Info] logWriterStat:: 0 FlushedCount 14280000 QueuedCount 49975
2016-04-06T08:20:08.225+05:30 [Info] logWriterStat:: 0 FlushedCount 14290000 QueuedCount 49961
2016-04-06T08:20:08.308+05:30 [Info] logWriterStat:: 0 FlushedCount 14300000 QueuedCount 49964
2016-04-06T08:20:08.383+05:30 [Info] logWriterStat:: 0 FlushedCount 14310000 QueuedCount 49986
2016-04-06T08:20:08.475+05:30 [Info] logWriterStat:: 0 FlushedCount 14320000 QueuedCount 49946
2016-04-06T08:20:08.562+05:30 [Info] logWriterStat:: 0 FlushedCount 14330000 QueuedCount 49936
2016-04-06T08:20:08.643+05:30 [Info] logWriterStat:: 0 FlushedCount 14340000 QueuedCount 49942
2016-04-06T08:20:08.727+05:30 [Info] logWriterStat:: 0 FlushedCount 14350000 QueuedCount 49945
2016-04-06T08:20:08.811+05:30 [Info] logWriterStat:: 0 FlushedCount 14360000 QueuedCount 49929
2016-04-06T08:20:08.895+05:30 [Info] logWriterStat:: 0 FlushedCount 14370000 QueuedCount 49957
2016-04-06T08:20:08.977+05:30 [Info] logWriterStat:: 0 FlushedCount 14380000 QueuedCount 49976
2016-04-06T08:20:09.07+05:30 [Info] logWriterStat:: 0 FlushedCount 14390000 QueuedCount 49987
2016-04-06T08:20:09.152+05:30 [Info] logWriterStat:: 0 FlushedCount 14400000 QueuedCount 49947
2016-04-06T08:20:09.234+05:30 [Info] logWriterStat:: 0 FlushedCount 14410000 QueuedCount 49959
2016-04-06T08:20:09.319+05:30 [Info] logWriterStat:: 0 FlushedCount 14420000 QueuedCount 49998
2016-04-06T08:20:09.405+05:30 [Info] logWriterStat:: 0 FlushedCount 14430000 QueuedCount 49988
2016-04-06T08:20:09.485+05:30 [Info] logWriterStat:: 0 FlushedCount 14440000 QueuedCount 46202
2016-04-06T08:20:09.559+05:30 [Info] logWriterStat:: 0 FlushedCount 14450000 QueuedCount 43602
2016-04-06T08:20:09.638+05:30 [Info] logWriterStat:: 0 FlushedCount 14460000 QueuedCount 43415
2016-04-06T08:20:09.719+05:30 [Info] logWriterStat:: 0 FlushedCount 14470000 QueuedCount 42681
2016-04-06T08:20:09.797+05:30 [Info] logWriterStat:: 0 FlushedCount 14480000 QueuedCount 41295
2016-04-06T08:20:09.871+05:30 [Info] logWriterStat:: 0 FlushedCount 14490000 QueuedCount 42482
2016-04-06T08:20:09.947+05:30 [Info] logWriterStat:: 0 FlushedCount 14500000 QueuedCount 41331
2016-04-06T08:20:10.023+05:30 [Info] logWriterStat:: 0 FlushedCount 14510000 QueuedCount 40633
2016-04-06T08:20:10.1+05:30 [Info] logWriterStat:: 0 FlushedCount 14520000 QueuedCount 39149
2016-04-06T08:20:10.18+05:30 [Info] logWriterStat:: 0 FlushedCount 14530000 QueuedCount 41262
2016-04-06T08:20:10.257+05:30 [Info] logWriterStat:: 0 FlushedCount 14540000 QueuedCount 39988
2016-04-06T08:20:10.332+05:30 [Info] logWriterStat:: 0 FlushedCount 14550000 QueuedCount 39330
2016-04-06T08:20:10.408+05:30 [Info] logWriterStat:: 0 FlushedCount 14560000 QueuedCount 37111
2016-04-06T08:20:10.52+05:30 [Info] logWriterStat:: 0 FlushedCount 14570000 QueuedCount 49845
2016-04-06T08:20:10.621+05:30 [Info] logWriterStat:: 0 FlushedCount 14580000 QueuedCount 49964
2016-04-06T08:20:10.699+05:30 [Info] logWriterStat:: 0 FlushedCount 14590000 QueuedCount 49931
2016-04-06T08:20:10.79+05:30 [Info] logWriterStat:: 0 FlushedCount 14600000 QueuedCount 49458
2016-04-06T08:20:10.873+05:30 [Info] logWriterStat:: 0 FlushedCount 14610000 QueuedCount 49857
2016-04-06T08:20:10.958+05:30 [Info] logWriterStat:: 0 FlushedCount 14620000 QueuedCount 49983
2016-04-06T08:20:11.038+05:30 [Info] logWriterStat:: 0 FlushedCount 14630000 QueuedCount 49994
2016-04-06T08:20:11.119+05:30 [Info] logWriterStat:: 0 FlushedCount 14640000 QueuedCount 49995
2016-04-06T08:20:11.202+05:30 [Info] logWriterStat:: 0 FlushedCount 14650000 QueuedCount 49861
2016-04-06T08:20:11.283+05:30 [Info] logWriterStat:: 0 FlushedCount 14660000 QueuedCount 49965
2016-04-06T08:20:11.363+05:30 [Info] logWriterStat:: 0 FlushedCount 14670000 QueuedCount 49895
2016-04-06T08:20:11.447+05:30 [Info] logWriterStat:: 0 FlushedCount 14680000 QueuedCount 49984
2016-04-06T08:20:11.535+05:30 [Info] logWriterStat:: 0 FlushedCount 14690000 QueuedCount 49980
2016-04-06T08:20:11.619+05:30 [Info] logWriterStat:: 0 FlushedCount 14700000 QueuedCount 49976
2016-04-06T08:20:11.703+05:30 [Info] logWriterStat:: 0 FlushedCount 14710000 QueuedCount 49959
2016-04-06T08:20:11.787+05:30 [Info] logWriterStat:: 0 FlushedCount 14720000 QueuedCount 49867
2016-04-06T08:20:11.869+05:30 [Info] logWriterStat:: 0 FlushedCount 14730000 QueuedCount 49999
2016-04-06T08:20:11.955+05:30 [Info] logWriterStat:: 0 FlushedCount 14740000 QueuedCount 49917
2016-04-06T08:20:12.037+05:30 [Info] logWriterStat:: 0 FlushedCount 14750000 QueuedCount 49903
2016-04-06T08:20:12.119+05:30 [Info] logWriterStat:: 0 FlushedCount 14760000 QueuedCount 49875
2016-04-06T08:20:12.198+05:30 [Info] logWriterStat:: 0 FlushedCount 14770000 QueuedCount 49963
2016-04-06T08:20:12.28+05:30 [Info] logWriterStat:: 0 FlushedCount 14780000 QueuedCount 49958
2016-04-06T08:20:12.358+05:30 [Info] logWriterStat:: 0 FlushedCount 14790000 QueuedCount 49871
2016-04-06T08:20:12.447+05:30 [Info] logWriterStat:: 0 FlushedCount 14800000 QueuedCount 49982
2016-04-06T08:20:12.529+05:30 [Info] logWriterStat:: 0 FlushedCount 14810000 QueuedCount 49965
2016-04-06T08:20:12.613+05:30 [Info] logWriterStat:: 0 FlushedCount 14820000 QueuedCount 49981
2016-04-06T08:20:12.706+05:30 [Info] logWriterStat:: 0 FlushedCount 14830000 QueuedCount 49992
2016-04-06T08:20:12.788+05:30 [Info] logWriterStat:: 0 FlushedCount 14840000 QueuedCount 49988
2016-04-06T08:20:12.871+05:30 [Info] logWriterStat:: 0 FlushedCount 14850000 QueuedCount 49975
2016-04-06T08:20:12.953+05:30 [Info] logWriterStat:: 0 FlushedCount 14860000 QueuedCount 49987
2016-04-06T08:20:13.036+05:30 [Info] logWriterStat:: 0 FlushedCount 14870000 QueuedCount 49992
2016-04-06T08:20:13.12+05:30 [Info] logWriterStat:: 0 FlushedCount 14880000 QueuedCount 49913
2016-04-06T08:20:13.196+05:30 [Info] logWriterStat:: 0 FlushedCount 14890000 QueuedCount 49955
2016-04-06T08:20:13.277+05:30 [Info] logWriterStat:: 0 FlushedCount 14900000 QueuedCount 49978
2016-04-06T08:20:13.362+05:30 [Info] logWriterStat:: 0 FlushedCount 14910000 QueuedCount 49999
2016-04-06T08:20:13.449+05:30 [Info] logWriterStat:: 0 FlushedCount 14920000 QueuedCount 49991
2016-04-06T08:20:13.53+05:30 [Info] logWriterStat:: 0 FlushedCount 14930000 QueuedCount 49941
2016-04-06T08:20:13.615+05:30 [Info] logWriterStat:: 0 FlushedCount 14940000 QueuedCount 49974
2016-04-06T08:20:13.7+05:30 [Info] logWriterStat:: 0 FlushedCount 14950000 QueuedCount 49915
2016-04-06T08:20:13.779+05:30 [Info] logWriterStat:: 0 FlushedCount 14960000 QueuedCount 49944
2016-04-06T08:20:13.86+05:30 [Info] logWriterStat:: 0 FlushedCount 14970000 QueuedCount 49987
2016-04-06T08:20:13.948+05:30 [Info] logWriterStat:: 0 FlushedCount 14980000 QueuedCount 49982
2016-04-06T08:20:14.037+05:30 [Info] logWriterStat:: 0 FlushedCount 14990000 QueuedCount 49975
2016-04-06T08:20:14.126+05:30 [Info] logWriterStat:: 0 FlushedCount 15000000 QueuedCount 49916
2016-04-06T08:20:14.213+05:30 [Info] logWriterStat:: 0 FlushedCount 15010000 QueuedCount 49996
2016-04-06T08:20:14.293+05:30 [Info] logWriterStat:: 0 FlushedCount 15020000 QueuedCount 49966
2016-04-06T08:20:14.37+05:30 [Info] logWriterStat:: 0 FlushedCount 15030000 QueuedCount 49969
2016-04-06T08:20:14.456+05:30 [Info] logWriterStat:: 0 FlushedCount 15040000 QueuedCount 49963
2016-04-06T08:20:14.538+05:30 [Info] logWriterStat:: 0 FlushedCount 15050000 QueuedCount 49975
2016-04-06T08:20:14.62+05:30 [Info] logWriterStat:: 0 FlushedCount 15060000 QueuedCount 49954
2016-04-06T08:20:14.702+05:30 [Info] logWriterStat:: 0 FlushedCount 15070000 QueuedCount 49961
2016-04-06T08:20:14.783+05:30 [Info] logWriterStat:: 0 FlushedCount 15080000 QueuedCount 49801
2016-04-06T08:20:14.864+05:30 [Info] logWriterStat:: 0 FlushedCount 15090000 QueuedCount 49974
2016-04-06T08:20:14.947+05:30 [Info] logWriterStat:: 0 FlushedCount 15100000 QueuedCount 49974
2016-04-06T08:20:15.026+05:30 [Info] logWriterStat:: 0 FlushedCount 15110000 QueuedCount 49874
2016-04-06T08:20:15.109+05:30 [Info] logWriterStat:: 0 FlushedCount 15120000 QueuedCount 49931
2016-04-06T08:20:15.192+05:30 [Info] logWriterStat:: 0 FlushedCount 15130000 QueuedCount 49973
2016-04-06T08:20:15.27+05:30 [Info] logWriterStat:: 0 FlushedCount 15140000 QueuedCount 49982
2016-04-06T08:20:15.352+05:30 [Info] logWriterStat:: 0 FlushedCount 15150000 QueuedCount 49869
2016-04-06T08:20:15.431+05:30 [Info] logWriterStat:: 0 FlushedCount 15160000 QueuedCount 49989
2016-04-06T08:20:15.514+05:30 [Info] logWriterStat:: 0 FlushedCount 15170000 QueuedCount 49958
2016-04-06T08:20:15.599+05:30 [Info] logWriterStat:: 0 FlushedCount 15180000 QueuedCount 49984
2016-04-06T08:20:15.688+05:30 [Info] logWriterStat:: 0 FlushedCount 15190000 QueuedCount 49912
2016-04-06T08:20:15.788+05:30 [Info] logWriterStat:: 0 FlushedCount 15200000 QueuedCount 49968
2016-04-06T08:20:15.873+05:30 [Info] logWriterStat:: 0 FlushedCount 15210000 QueuedCount 49983
2016-04-06T08:20:15.957+05:30 [Info] logWriterStat:: 0 FlushedCount 15220000 QueuedCount 49976
2016-04-06T08:20:16.038+05:30 [Info] logWriterStat:: 0 FlushedCount 15230000 QueuedCount 49918
2016-04-06T08:20:16.116+05:30 [Info] logWriterStat:: 0 FlushedCount 15240000 QueuedCount 49525
2016-04-06T08:20:16.201+05:30 [Info] logWriterStat:: 0 FlushedCount 15250000 QueuedCount 48910
2016-04-06T08:20:16.282+05:30 [Info] logWriterStat:: 0 FlushedCount 15260000 QueuedCount 48247
2016-04-06T08:20:16.361+05:30 [Info] logWriterStat:: 0 FlushedCount 15270000 QueuedCount 47350
2016-04-06T08:20:16.452+05:30 [Info] logWriterStat:: 0 FlushedCount 15280000 QueuedCount 46849
2016-04-06T08:20:16.534+05:30 [Info] logWriterStat:: 0 FlushedCount 15290000 QueuedCount 46841
2016-04-06T08:20:16.622+05:30 [Info] logWriterStat:: 0 FlushedCount 15300000 QueuedCount 46866
2016-04-06T08:20:16.705+05:30 [Info] logWriterStat:: 0 FlushedCount 15310000 QueuedCount 46864
2016-04-06T08:20:16.788+05:30 [Info] logWriterStat:: 0 FlushedCount 15320000 QueuedCount 46848
2016-04-06T08:20:16.867+05:30 [Info] logWriterStat:: 0 FlushedCount 15330000 QueuedCount 46858
2016-04-06T08:20:16.949+05:30 [Info] logWriterStat:: 0 FlushedCount 15340000 QueuedCount 46870
2016-04-06T08:20:17.033+05:30 [Info] logWriterStat:: 0 FlushedCount 15350000 QueuedCount 46827
2016-04-06T08:20:17.111+05:30 [Info] logWriterStat:: 0 FlushedCount 15360000 QueuedCount 46862
2016-04-06T08:20:17.196+05:30 [Info] logWriterStat:: 0 FlushedCount 15370000 QueuedCount 46809
2016-04-06T08:20:17.282+05:30 [Info] logWriterStat:: 0 FlushedCount 15380000 QueuedCount 46863
2016-04-06T08:20:17.367+05:30 [Info] logWriterStat:: 0 FlushedCount 15390000 QueuedCount 46851
2016-04-06T08:20:17.452+05:30 [Info] logWriterStat:: 0 FlushedCount 15400000 QueuedCount 46677
2016-04-06T08:20:17.52+05:30 [Info] logWriterStat:: 0 FlushedCount 15410000 QueuedCount 46816
2016-04-06T08:20:17.606+05:30 [Info] logWriterStat:: 0 FlushedCount 15420000 QueuedCount 46856
2016-04-06T08:20:17.692+05:30 [Info] logWriterStat:: 0 FlushedCount 15430000 QueuedCount 46834
2016-04-06T08:20:17.79+05:30 [Info] logWriterStat:: 0 FlushedCount 15440000 QueuedCount 46692
2016-04-06T08:20:17.864+05:30 [Info] logWriterStat:: 0 FlushedCount 15450000 QueuedCount 46696
2016-04-06T08:20:17.945+05:30 [Info] logWriterStat:: 0 FlushedCount 15460000 QueuedCount 46794
2016-04-06T08:20:18.026+05:30 [Info] logWriterStat:: 0 FlushedCount 15470000 QueuedCount 46860
2016-04-06T08:20:18.111+05:30 [Info] logWriterStat:: 0 FlushedCount 15480000 QueuedCount 46810
2016-04-06T08:20:18.194+05:30 [Info] logWriterStat:: 0 FlushedCount 15490000 QueuedCount 46860
2016-04-06T08:20:18.276+05:30 [Info] logWriterStat:: 0 FlushedCount 15500000 QueuedCount 46842
2016-04-06T08:20:18.358+05:30 [Info] logWriterStat:: 0 FlushedCount 15510000 QueuedCount 46769
2016-04-06T08:20:18.448+05:30 [Info] logWriterStat:: 0 FlushedCount 15520000 QueuedCount 46843
2016-04-06T08:20:18.531+05:30 [Info] logWriterStat:: 0 FlushedCount 15530000 QueuedCount 46779
2016-04-06T08:20:18.615+05:30 [Info] logWriterStat:: 0 FlushedCount 15540000 QueuedCount 46845
2016-04-06T08:20:18.706+05:30 [Info] logWriterStat:: 0 FlushedCount 15550000 QueuedCount 46862
2016-04-06T08:20:18.787+05:30 [Info] logWriterStat:: 0 FlushedCount 15560000 QueuedCount 46751
2016-04-06T08:20:18.87+05:30 [Info] logWriterStat:: 0 FlushedCount 15570000 QueuedCount 46812
2016-04-06T08:20:18.958+05:30 [Info] logWriterStat:: 0 FlushedCount 15580000 QueuedCount 46863
2016-04-06T08:20:19.04+05:30 [Info] logWriterStat:: 0 FlushedCount 15590000 QueuedCount 46829
2016-04-06T08:20:19.132+05:30 [Info] logWriterStat:: 0 FlushedCount 15600000 QueuedCount 46839
2016-04-06T08:20:19.214+05:30 [Info] logWriterStat:: 0 FlushedCount 15610000 QueuedCount 46803
2016-04-06T08:20:19.294+05:30 [Info] logWriterStat:: 0 FlushedCount 15620000 QueuedCount 46658
2016-04-06T08:20:19.377+05:30 [Info] logWriterStat:: 0 FlushedCount 15630000 QueuedCount 46090
2016-04-06T08:20:19.471+05:30 [Info] logWriterStat:: 0 FlushedCount 15640000 QueuedCount 45382
2016-04-06T08:20:19.553+05:30 [Info] logWriterStat:: 0 FlushedCount 15650000 QueuedCount 44274
2016-04-06T08:20:19.638+05:30 [Info] logWriterStat:: 0 FlushedCount 15660000 QueuedCount 42850
2016-04-06T08:20:19.722+05:30 [Info] logWriterStat:: 0 FlushedCount 15670000 QueuedCount 41931
2016-04-06T08:20:19.803+05:30 [Info] logWriterStat:: 0 FlushedCount 15680000 QueuedCount 41146
2016-04-06T08:20:19.886+05:30 [Info] logWriterStat:: 0 FlushedCount 15690000 QueuedCount 40574
2016-04-06T08:20:19.965+05:30 [Info] logWriterStat:: 0 FlushedCount 15700000 QueuedCount 40609
2016-04-06T08:20:20.051+05:30 [Info] logWriterStat:: 0 FlushedCount 15710000 QueuedCount 40602
2016-04-06T08:20:20.136+05:30 [Info] logWriterStat:: 0 FlushedCount 15720000 QueuedCount 40234
2016-04-06T08:20:20.219+05:30 [Info] logWriterStat:: 0 FlushedCount 15730000 QueuedCount 39535
2016-04-06T08:20:20.292+05:30 [Info] logWriterStat:: 0 FlushedCount 15740000 QueuedCount 38440
2016-04-06T08:20:20.375+05:30 [Info] logWriterStat:: 0 FlushedCount 15750000 QueuedCount 37673
2016-04-06T08:20:20.485+05:30 [Info] logWriterStat:: 0 FlushedCount 15760000 QueuedCount 37379
2016-04-06T08:20:20.592+05:30 [Info] logWriterStat:: 0 FlushedCount 15770000 QueuedCount 35916
2016-04-06T08:20:20.675+05:30 [Info] logWriterStat:: 0 FlushedCount 15780000 QueuedCount 34401
2016-04-06T08:20:20.761+05:30 [Info] logWriterStat:: 0 FlushedCount 15790000 QueuedCount 32456
2016-04-06T08:20:20.84+05:30 [Info] logWriterStat:: 0 FlushedCount 15800000 QueuedCount 31150
2016-04-06T08:20:20.92+05:30 [Info] logWriterStat:: 0 FlushedCount 15810000 QueuedCount 31198
2016-04-06T08:20:21.007+05:30 [Info] logWriterStat:: 0 FlushedCount 15820000 QueuedCount 31238
2016-04-06T08:20:21.114+05:30 [Info] logWriterStat:: 0 FlushedCount 15830000 QueuedCount 31125
2016-04-06T08:20:21.193+05:30 [Info] logWriterStat:: 0 FlushedCount 15840000 QueuedCount 31174
2016-04-06T08:20:21.266+05:30 [Info] logWriterStat:: 0 FlushedCount 15850000 QueuedCount 31054
2016-04-06T08:20:21.349+05:30 [Info] logWriterStat:: 0 FlushedCount 15860000 QueuedCount 30033
2016-04-06T08:20:21.428+05:30 [Info] logWriterStat:: 0 FlushedCount 15870000 QueuedCount 29129
2016-04-06T08:20:21.515+05:30 [Info] logWriterStat:: 0 FlushedCount 15880000 QueuedCount 28088
2016-04-06T08:20:21.592+05:30 [Info] logWriterStat:: 0 FlushedCount 15890000 QueuedCount 27776
2016-04-06T08:20:21.667+05:30 [Info] logWriterStat:: 0 FlushedCount 15900000 QueuedCount 26397
2016-04-06T08:20:21.753+05:30 [Info] logWriterStat:: 0 FlushedCount 15910000 QueuedCount 24089
2016-04-06T08:20:21.833+05:30 [Info] logWriterStat:: 0 FlushedCount 15920000 QueuedCount 22217
2016-04-06T08:20:21.905+05:30 [Info] logWriterStat:: 0 FlushedCount 15930000 QueuedCount 21833
2016-04-06T08:20:21.981+05:30 [Info] logWriterStat:: 0 FlushedCount 15940000 QueuedCount 21479
2016-04-06T08:20:22.048+05:30 [Info] logWriterStat:: 0 FlushedCount 15950000 QueuedCount 20365
2016-04-06T08:20:22.124+05:30 [Info] logWriterStat:: 0 FlushedCount 15960000 QueuedCount 18220
2016-04-06T08:20:22.193+05:30 [Info] logWriterStat:: 0 FlushedCount 15970000 QueuedCount 16751
2016-04-06T08:20:22.26+05:30 [Info] logWriterStat:: 0 FlushedCount 15980000 QueuedCount 14546
2016-04-06T08:20:22.326+05:30 [Info] logWriterStat:: 0 FlushedCount 15990000 QueuedCount 11838
2016-04-06T08:20:22.394+05:30 [Info] logWriterStat:: 0 FlushedCount 16000000 QueuedCount 9351
2016-04-06T08:20:22.476+05:30 [Info] logWriterStat:: 0 FlushedCount 16010000 QueuedCount 6412
2016-04-06T08:20:22.608+05:30 [Info] logWriterStat:: 0 FlushedCount 16020000 QueuedCount 3118
2016-04-06T08:20:22.8+05:30 [Info] logWriterStat:: 0 FlushedCount 16030000 QueuedCount 2970
2016-04-06T08:20:22.852+05:30 [Info] MemDBSlice::OpenSnapshot SliceId 0 IndexInstId 0 Creating New Snapshot SnapshotInfo: count:10000000 committed:false
Initial build: 10000000 items took 1m27.695077004s -> 114031.48661975488 items/s
Incr build: 10000000 items took 1m4.850492321s -> 154200.83398136025 items/s
Main Index: node_count             = 16032970
soft_deletes           = 0
read_conflicts         = 0
insert_conflicts       = 0
next_pointers_per_node = NaN
memory_used            = 1510462518
node_allocs            = 16032970
node_frees             = 0

level_node_distribution:
level0 => 12025638
level1 => 3006119
level2 => 750698
level3 => 187828
level4 => 47006
level5 => 11841
level6 => 2889
level7 => 700
level8 => 196
level9 => 36
level10 => 16
level11 => 2
level12 => 1
level13 => 0
level14 => 0
level15 => 0
level16 => 0
level17 => 0
level18 => 0
level19 => 0
level20 => 0
level21 => 0
level22 => 0
level23 => 0
level24 => 0
level25 => 0
level26 => 0
level27 => 0
level28 => 0
level29 => 0
level30 => 0
level31 => 0
level32 => 0

Back Index 0 : 
FastHTCount = 625000
SlowHTCount = 0
Conflicts   = 0
MemoryInUse = 26250000

Back Index 1 : 
FastHTCount = 625000
SlowHTCount = 0
Conflicts   = 0
MemoryInUse = 26250000

Back Index 2 : 
FastHTCount = 625000
SlowHTCount = 0
Conflicts   = 0
MemoryInUse = 26250000

Back Index 3 : 
FastHTCount = 625000
SlowHTCount = 0
Conflicts   = 0
MemoryInUse = 26250000

Back Index 4 : 
FastHTCount = 625000
SlowHTCount = 0
Conflicts   = 0
MemoryInUse = 26250000

Back Index 5 : 
FastHTCount = 625000
SlowHTCount = 0
Conflicts   = 0
MemoryInUse = 26250000

Back Index 6 : 
FastHTCount = 625000
SlowHTCount = 0
Conflicts   = 0
MemoryInUse = 26250000

Back Index 7 : 
FastHTCount = 625000
SlowHTCount = 0
Conflicts   = 0
MemoryInUse = 26250000

Back Index 8 : 
FastHTCount = 625000
SlowHTCount = 0
Conflicts   = 0
MemoryInUse = 26250000

Back Index 9 : 
FastHTCount = 625000
SlowHTCount = 0
Conflicts   = 0
MemoryInUse = 26250000

Back Index 10 : 
FastHTCount = 625000
SlowHTCount = 0
Conflicts   = 0
MemoryInUse = 26250000

Back Index 11 : 
FastHTCount = 625000
SlowHTCount = 0
Conflicts   = 0
MemoryInUse = 26250000

Back Index 12 : 
FastHTCount = 625000
SlowHTCount = 0
Conflicts   = 0
MemoryInUse = 26250000

Back Index 13 : 
FastHTCount = 625000
SlowHTCount = 0
Conflicts   = 0
MemoryInUse = 26250000

Back Index 14 : 
FastHTCount = 625000
SlowHTCount = 0
Conflicts   = 0
MemoryInUse = 26250000

Back Index 15 : 
FastHTCount = 625000
SlowHTCount = 0
Conflicts   = 0
MemoryInUse = 26250000

--- PASS: TestMemDBInsertionPerf (152.55s)
PASS
ok  	github.com/couchbase/indexing/secondary/indexer	153.015s

Functional tests

2016/04/06 08:20:28 In init()
2016/04/06 08:20:28 Changing config key indexer.settings.persisted_snapshot_init_build.moi.interval to value 60000
2016/04/06 08:20:28 Type of settings value is float64
2016/04/06 08:20:28 Type of config value that was updated is float64
2016/04/06 08:20:28 After updation, Setting  is indexer.settings.persisted_snapshot_init_build.moi.interval = 60000
2016/04/06 08:20:28 Settings updated indexer.settings.persisted_snapshot_init_build.moi.interval = 60000
2016/04/06 08:20:28 Changing config key indexer.settings.persisted_snapshot.moi.interval to value 60000
2016/04/06 08:20:28 Type of settings value is float64
2016/04/06 08:20:28 Type of config value that was updated is float64
2016/04/06 08:20:28 After updation, Setting  is indexer.settings.persisted_snapshot.moi.interval = 60000
2016/04/06 08:20:28 Settings updated indexer.settings.persisted_snapshot.moi.interval = 60000
2016/04/06 08:20:28 Using forestdb for creating indexes
2016/04/06 08:20:28 Changing config key indexer.settings.storage_mode to value forestdb
2016/04/06 08:20:28 Error in ChangeIndexerSettings: Post http://127.0.0.1:9108/settings: EOF
panic: Error in ChangeIndexerSettings: Post http://127.0.0.1:9108/settings: EOF


goroutine 1 [running]:
runtime.gopanic(0xb51320, 0xc82019de80)
	/usr/local/go/src/runtime/panic.go:461 +0x3e6 fp=0xc8200e7b58 sp=0xc8200e7ad8
log.Panicf(0xe67ee8, 0x7, 0xc8200e7c18, 0x2, 0x2)
	/usr/local/go/src/log/log.go:327 +0xd8 fp=0xc8200e7bb8 sp=0xc8200e7b58
github.com/couchbase/indexing/secondary/tests/framework/common.HandleError(0x7fab15ef81d0, 0xc8202c2660, 0xf0bb40, 0x1e)
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/framework/common/util.go:41 +0x154 fp=0xc8200e7c40 sp=0xc8200e7bb8
github.com/couchbase/indexing/secondary/tests/functionaltests.init.1()
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/functionaltests/common_test.go:65 +0x7a4 fp=0xc8200e7e70 sp=0xc8200e7c40
github.com/couchbase/indexing/secondary/tests/functionaltests.init()
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/functionaltests/seti_n1qlclient_test.go:383 +0x106e fp=0xc8200e7f38 sp=0xc8200e7e70
main.init()
	github.com/couchbase/indexing/secondary/tests/functionaltests/_test/_testmain.go:260 +0x4a fp=0xc8200e7f40 sp=0xc8200e7f38
runtime.main()
	/usr/local/go/src/runtime/proc.go:100 +0x27f fp=0xc8200e7f90 sp=0xc8200e7f40
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1721 +0x1 fp=0xc8200e7f98 sp=0xc8200e7f90

goroutine 17 [syscall, locked to thread]:
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1721 +0x1 fp=0xc82003afb8 sp=0xc82003afb0

goroutine 2 [force gc (idle)]:
runtime.gopark(0x104e980, 0x160a0a0, 0xe83310, 0xf, 0x14, 0x1)
	/usr/local/go/src/runtime/proc.go:185 +0x163 fp=0xc820026758 sp=0xc820026730
runtime.goparkunlock(0x160a0a0, 0xe83310, 0xf, 0xc820000314, 0x1)
	/usr/local/go/src/runtime/proc.go:191 +0x54 fp=0xc820026790 sp=0xc820026758
runtime.forcegchelper()
	/usr/local/go/src/runtime/proc.go:152 +0xb8 fp=0xc8200267c0 sp=0xc820026790
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1721 +0x1 fp=0xc8200267c8 sp=0xc8200267c0
created by runtime.init.4
	/usr/local/go/src/runtime/proc.go:141 +0x2b

goroutine 3 [GC sweep wait]:
runtime.gopark(0x104e980, 0x160a300, 0xe77ea0, 0xd, 0x452714, 0x1)
	/usr/local/go/src/runtime/proc.go:185 +0x163 fp=0xc820026f48 sp=0xc820026f20
runtime.goparkunlock(0x160a300, 0xe77ea0, 0xd, 0x14, 0x1)
	/usr/local/go/src/runtime/proc.go:191 +0x54 fp=0xc820026f80 sp=0xc820026f48
runtime.bgsweep(0xc820056000)
	/usr/local/go/src/runtime/mgcsweep.go:67 +0x14d fp=0xc820026fb8 sp=0xc820026f80
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1721 +0x1 fp=0xc820026fc0 sp=0xc820026fb8
created by runtime.gcenable
	/usr/local/go/src/runtime/mgc.go:206 +0x53

goroutine 4 [finalizer wait]:
runtime.gopark(0x104e980, 0x1630f80, 0xe82e80, 0xe, 0x100000014, 0x1)
	/usr/local/go/src/runtime/proc.go:185 +0x163 fp=0xc820027718 sp=0xc8200276f0
runtime.goparkunlock(0x1630f80, 0xe82e80, 0xe, 0x2000000014, 0x1)
	/usr/local/go/src/runtime/proc.go:191 +0x54 fp=0xc820027750 sp=0xc820027718
runtime.runfinq()
	/usr/local/go/src/runtime/mfinal.go:154 +0xaa fp=0xc8200277c0 sp=0xc820027750
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1721 +0x1 fp=0xc8200277c8 sp=0xc8200277c0
created by runtime.createfing
	/usr/local/go/src/runtime/mfinal.go:135 +0x60

goroutine 6 [IO wait]:
runtime.gopark(0x104e940, 0x7fab15ef2688, 0xe69a28, 0x7, 0x4fc71b, 0x5)
	/usr/local/go/src/runtime/proc.go:185 +0x163 fp=0xc82004d2f0 sp=0xc82004d2c8
runtime.netpollblock(0x7fab15ef2660, 0xffffff0000000072, 0x0)
	/usr/local/go/src/runtime/netpoll.go:338 +0x15e fp=0xc82004d338 sp=0xc82004d2f0
net.runtime_pollWait(0x7fab15ef2660, 0x72, 0xc82000e160)
	/usr/local/go/src/runtime/netpoll.go:157 +0x60 fp=0xc82004d358 sp=0xc82004d338
net.(*pollDesc).Wait(0xc8200566f0, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x3a fp=0xc82004d378 sp=0xc82004d358
net.(*pollDesc).WaitRead(0xc8200566f0, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x36 fp=0xc82004d3a0 sp=0xc82004d378
net.(*netFD).Read(0xc820056690, 0xc8202e8000, 0x1000, 0x1000, 0x0, 0x7fab15eed050, 0xc82000e160)
	/usr/local/go/src/net/fd_unix.go:232 +0x23a fp=0xc82004d470 sp=0xc82004d3a0
net.(*conn).Read(0xc82002a0c8, 0xc8202e8000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:172 +0xe4 fp=0xc82004d4d8 sp=0xc82004d470
net/http.noteEOFReader.Read(0x7fab15ef2be0, 0xc82002a0c8, 0xc8200c34f8, 0xc8202e8000, 0x1000, 0x1000, 0x3c, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1370 +0x67 fp=0xc82004d528 sp=0xc82004d4d8
net/http.(*noteEOFReader).Read(0xc82000d900, 0xc8202e8000, 0x1000, 0x1000, 0x13c6, 0x0, 0x0)
	:126 +0xd0 fp=0xc82004d578 sp=0xc82004d528
bufio.(*Reader).fill(0xc820014900)
	/usr/local/go/src/bufio/bufio.go:97 +0x1e9 fp=0xc82004d638 sp=0xc82004d578
bufio.(*Reader).ReadSlice(0xc820014900, 0xa, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:328 +0x21a fp=0xc82004d680 sp=0xc82004d638
net/http/internal.readLine(0xc820014900, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/internal/chunked.go:110 +0x4b fp=0xc82004d6c0 sp=0xc82004d680
net/http/internal.(*chunkedReader).beginChunk(0xc8202c2540)
	/usr/local/go/src/net/http/internal/chunked.go:47 +0x39 fp=0xc82004d720 sp=0xc82004d6c0
net/http/internal.(*chunkedReader).Read(0xc8202c2540, 0xc82030c000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/internal/chunked.go:77 +0xb7 fp=0xc82004d7e8 sp=0xc82004d720
net/http.(*body).readLocked(0xc8202de240, 0xc82030c000, 0x1000, 0x1000, 0xc800000000, 0x0, 0x0)
	/usr/local/go/src/net/http/transfer.go:632 +0x98 fp=0xc82004d838 sp=0xc82004d7e8
net/http.(*body).Read(0xc8202de240, 0xc82030c000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transfer.go:624 +0x110 fp=0xc82004d888 sp=0xc82004d838
net/http.(*bodyEOFSignal).Read(0xc8202de280, 0xc82030c000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1296 +0x26a fp=0xc82004d928 sp=0xc82004d888
bufio.(*Reader).fill(0xc82004dd68)
	/usr/local/go/src/bufio/bufio.go:97 +0x1e9 fp=0xc82004d9e8 sp=0xc82004d928
bufio.(*Reader).ReadSlice(0xc82004dd68, 0xc82030c40a, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:328 +0x21a fp=0xc82004da30 sp=0xc82004d9e8
bufio.(*Reader).ReadBytes(0xc82004dd68, 0xc82007ea0a, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:406 +0xa9 fp=0xc82004db50 sp=0xc82004da30
github.com/couchbase/indexing/secondary/dcp.(*Client).runObserveStreamingEndpoint(0xc8200c3430, 0xc82000d860, 0x17, 0x104a9a8, 0xc82000d840, 0x0, 0x0, 0x0)
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/dcp/pools.go:366 +0x806 fp=0xc82004de98 sp=0xc82004db50
github.com/couchbase/indexing/secondary/dcp.(*Client).RunObservePool(0xc8200c3430, 0xe6d388, 0x7, 0xc82000d840, 0x0, 0x0, 0x0)
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/dcp/pools.go:305 +0xab fp=0xc82004def0 sp=0xc82004de98
github.com/couchbase/indexing/secondary/common.(*serviceNotifierInstance).RunPoolObserver(0xc8200c33f0)
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/common/services_notifier.go:78 +0x9a fp=0xc82004df98 sp=0xc82004def0
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1721 +0x1 fp=0xc82004dfa0 sp=0xc82004df98
created by github.com/couchbase/indexing/secondary/common.NewServicesChangeNotifier
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/common/services_notifier.go:156 +0x53b

goroutine 19 [syscall]:
runtime.notetsleepg(0x1631300, 0xffffffffffffffff, 0x1)
	/usr/local/go/src/runtime/lock_futex.go:202 +0x4e fp=0xc820022740 sp=0xc820022718
runtime.signal_recv(0x0)
	/usr/local/go/src/runtime/sigqueue.go:111 +0x132 fp=0xc820022778 sp=0xc820022740
os/signal.loop()
	/usr/local/go/src/os/signal/signal_unix.go:22 +0x18 fp=0xc8200227c0 sp=0xc820022778
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1721 +0x1 fp=0xc8200227c8 sp=0xc8200227c0
created by os/signal.init.1
	/usr/local/go/src/os/signal/signal_unix.go:28 +0x37

goroutine 20 [syscall]:
runtime.notetsleepg(0x160a4f8, 0x5f590d3, 0x1)
	/usr/local/go/src/runtime/lock_futex.go:202 +0x4e fp=0xc820022f38 sp=0xc820022f10
runtime.timerproc()
	/usr/local/go/src/runtime/time.go:209 +0xde fp=0xc820022fc0 sp=0xc820022f38
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1721 +0x1 fp=0xc820022fc8 sp=0xc820022fc0
created by runtime.addtimerLocked
	/usr/local/go/src/runtime/time.go:116 +0x11f

goroutine 21 [IO wait]:
runtime.gopark(0x104e940, 0x7fab15ef2a48, 0xe69a28, 0x7, 0x4fc71b, 0x5)
	/usr/local/go/src/runtime/proc.go:185 +0x163 fp=0xc8202e5378 sp=0xc8202e5350
runtime.netpollblock(0x7fab15ef2a20, 0xffffff0000000072, 0x0)
	/usr/local/go/src/runtime/netpoll.go:338 +0x15e fp=0xc8202e53c0 sp=0xc8202e5378
net.runtime_pollWait(0x7fab15ef2a20, 0x72, 0xc82000e160)
	/usr/local/go/src/runtime/netpoll.go:157 +0x60 fp=0xc8202e53e0 sp=0xc8202e53c0
net.(*pollDesc).Wait(0xc820056140, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x3a fp=0xc8202e5400 sp=0xc8202e53e0
net.(*pollDesc).WaitRead(0xc820056140, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x36 fp=0xc8202e5428 sp=0xc8202e5400
net.(*netFD).Read(0xc8200560e0, 0xc82017f000, 0x1000, 0x1000, 0x0, 0x7fab15eed050, 0xc82000e160)
	/usr/local/go/src/net/fd_unix.go:232 +0x23a fp=0xc8202e54f8 sp=0xc8202e5428
net.(*conn).Read(0xc82002a0a0, 0xc82017f000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:172 +0xe4 fp=0xc8202e5560 sp=0xc8202e54f8
bufio.(*Reader).fill(0xc8200142a0)
	/usr/local/go/src/bufio/bufio.go:97 +0x1e9 fp=0xc8202e5620 sp=0xc8202e5560
bufio.(*Reader).Read(0xc8200142a0, 0xc82019a001, 0x5ff, 0x5ff, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:207 +0x260 fp=0xc8202e56d0 sp=0xc8202e5620
github.com/couchbase/cbauth/revrpc.(*minirwc).Read(0xc82000d140, 0xc82019a001, 0x5ff, 0x5ff, 0xc82019c120, 0x0, 0x0)
	/opt/build/godeps/src/github.com/couchbase/cbauth/revrpc/revrpc.go:100 +0x51 fp=0xc8202e5710 sp=0xc8202e56d0
encoding/json.(*Decoder).refill(0xc820196000, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:152 +0x287 fp=0xc8202e57c8 sp=0xc8202e5710
encoding/json.(*Decoder).readValue(0xc820196000, 0x1, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:128 +0x41b fp=0xc8202e5878 sp=0xc8202e57c8
encoding/json.(*Decoder).Decode(0xc820196000, 0xcca420, 0xc820124800, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:57 +0x159 fp=0xc8202e58f8 sp=0xc8202e5878
net/rpc/jsonrpc.(*serverCodec).ReadRequestHeader(0xc8201247e0, 0xc820104b20, 0x0, 0x0)
	/usr/local/go/src/net/rpc/jsonrpc/server.go:66 +0x7c fp=0xc8202e5948 sp=0xc8202e58f8
net/rpc.(*Server).readRequestHeader(0xc820106500, 0x7fab1408b700, 0xc8201247e0, 0x0, 0x0, 0xc820104b20, 0xc8200e3a00, 0x0, 0x0)
	/usr/local/go/src/net/rpc/server.go:576 +0x90 fp=0xc8202e5a50 sp=0xc8202e5948
net/rpc.(*Server).readRequest(0xc820106500, 0x7fab1408b700, 0xc8201247e0, 0xc820106500, 0xc82010ffd0, 0xc820126380, 0x0, 0x0, 0x0, 0x0, ...)
	/usr/local/go/src/net/rpc/server.go:543 +0x8b fp=0xc8202e5ac8 sp=0xc8202e5a50
net/rpc.(*Server).ServeCodec(0xc820106500, 0x7fab1408b700, 0xc8201247e0)
	/usr/local/go/src/net/rpc/server.go:462 +0x8c fp=0xc8202e5c08 sp=0xc8202e5ac8
github.com/couchbase/cbauth/revrpc.(*Service).Run(0xc820108690, 0xc8202e5f40, 0x0, 0x0)
	/opt/build/godeps/src/github.com/couchbase/cbauth/revrpc/revrpc.go:146 +0x9eb fp=0xc8202e5e90 sp=0xc8202e5c08
github.com/couchbase/cbauth/revrpc.BabysitService(0xc8202e5f40, 0xc820108690, 0x7fab1407f028, 0xc82000d040, 0x0, 0x0)
	/opt/build/godeps/src/github.com/couchbase/cbauth/revrpc/revrpc.go:242 +0x77 fp=0xc8202e5ee0 sp=0xc8202e5e90
github.com/couchbase/cbauth.runRPCForSvc(0xc820108690, 0xc820106240, 0x0, 0x0)
	/opt/build/godeps/src/github.com/couchbase/cbauth/default.go:56 +0x10a fp=0xc8202e5f58 sp=0xc8202e5ee0
github.com/couchbase/cbauth.startDefault.func1(0xc820108690, 0xc820106240)
	/opt/build/godeps/src/github.com/couchbase/cbauth/default.go:63 +0x2b fp=0xc8202e5f90 sp=0xc8202e5f58
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1721 +0x1 fp=0xc8202e5f98 sp=0xc8202e5f90
created by github.com/couchbase/cbauth.startDefault
	/opt/build/godeps/src/github.com/couchbase/cbauth/default.go:64 +0x120

goroutine 22 [sleep]:
runtime.gopark(0x104e980, 0x160a4e0, 0xe711b0, 0x5, 0x13, 0x2)
	/usr/local/go/src/runtime/proc.go:185 +0x163 fp=0xc820023eb8 sp=0xc820023e90
runtime.goparkunlock(0x160a4e0, 0xe711b0, 0x5, 0x13, 0x2)
	/usr/local/go/src/runtime/proc.go:191 +0x54 fp=0xc820023ef0 sp=0xc820023eb8
time.Sleep(0x2540be400)
	/usr/local/go/src/runtime/time.go:59 +0xf9 fp=0xc820023f38 sp=0xc820023ef0
github.com/couchbase/indexing/secondary/common.pollForDeletedBuckets()
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/common/dcp_seqno.go:341 +0x3e fp=0xc820023fc0 sp=0xc820023f38
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1721 +0x1 fp=0xc820023fc8 sp=0xc820023fc0
created by github.com/couchbase/indexing/secondary/common.init.1
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/common/dcp_seqno.go:34 +0xf3

goroutine 29 [select]:
runtime.gopark(0x104e9b8, 0xc8202d1ed8, 0xe70eb8, 0x6, 0x18, 0x2)
	/usr/local/go/src/runtime/proc.go:185 +0x163 fp=0xc8202d1b98 sp=0xc8202d1b70
runtime.selectgoImpl(0xc8202d1ed8, 0x0, 0x18)
	/usr/local/go/src/runtime/select.go:392 +0xa64 fp=0xc8202d1d40 sp=0xc8202d1b98
runtime.selectgo(0xc8202d1ed8)
	/usr/local/go/src/runtime/select.go:212 +0x12 fp=0xc8202d1d60 sp=0xc8202d1d40
net/http.(*persistConn).readLoop(0xc8201104d0)
	/usr/local/go/src/net/http/transport.go:976 +0xac7 fp=0xc8202d1fa8 sp=0xc8202d1d60
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1721 +0x1 fp=0xc8202d1fb0 sp=0xc8202d1fa8
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:685 +0xc78

goroutine 25 [IO wait]:
runtime.gopark(0x104e940, 0x7fab15ef2988, 0xe69a28, 0x7, 0x4fc71b, 0x5)
	/usr/local/go/src/runtime/proc.go:185 +0x163 fp=0xc820036a08 sp=0xc8200369e0
runtime.netpollblock(0x7fab15ef2960, 0xffffff0000000072, 0x0)
	/usr/local/go/src/runtime/netpoll.go:338 +0x15e fp=0xc820036a50 sp=0xc820036a08
net.runtime_pollWait(0x7fab15ef2960, 0x72, 0xc82000e160)
	/usr/local/go/src/runtime/netpoll.go:157 +0x60 fp=0xc820036a70 sp=0xc820036a50
net.(*pollDesc).Wait(0xc8201146f0, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x3a fp=0xc820036a90 sp=0xc820036a70
net.(*pollDesc).WaitRead(0xc8201146f0, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x36 fp=0xc820036ab8 sp=0xc820036a90
net.(*netFD).Read(0xc820114690, 0xc82018f000, 0x1000, 0x1000, 0x0, 0x7fab15eed050, 0xc82000e160)
	/usr/local/go/src/net/fd_unix.go:232 +0x23a fp=0xc820036b88 sp=0xc820036ab8
net.(*conn).Read(0xc82010c0a0, 0xc82018f000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:172 +0xe4 fp=0xc820036bf0 sp=0xc820036b88
net/http.noteEOFReader.Read(0x7fab15ef2be0, 0xc82010c0a0, 0xc820110268, 0xc82018f000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1370 +0x67 fp=0xc820036c40 sp=0xc820036bf0
net/http.(*noteEOFReader).Read(0xc820104d60, 0xc82018f000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	:126 +0xd0 fp=0xc820036c90 sp=0xc820036c40
bufio.(*Reader).fill(0xc820124ba0)
	/usr/local/go/src/bufio/bufio.go:97 +0x1e9 fp=0xc820036d50 sp=0xc820036c90
bufio.(*Reader).Peek(0xc820124ba0, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xcc fp=0xc820036d60 sp=0xc820036d50
net/http.(*persistConn).readLoop(0xc820110210)
	/usr/local/go/src/net/http/transport.go:876 +0xf7 fp=0xc820036fa8 sp=0xc820036d60
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1721 +0x1 fp=0xc820036fb0 sp=0xc820036fa8
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:685 +0xc78

goroutine 26 [select]:
runtime.gopark(0x104e9b8, 0xc820037f10, 0xe70eb8, 0x6, 0x18, 0x2)
	/usr/local/go/src/runtime/proc.go:185 +0x163 fp=0xc820037c78 sp=0xc820037c50
runtime.selectgoImpl(0xc820037f10, 0x0, 0x18)
	/usr/local/go/src/runtime/select.go:392 +0xa64 fp=0xc820037e20 sp=0xc820037c78
runtime.selectgo(0xc820037f10)
	/usr/local/go/src/runtime/select.go:212 +0x12 fp=0xc820037e40 sp=0xc820037e20
net/http.(*persistConn).writeLoop(0xc820110210)
	/usr/local/go/src/net/http/transport.go:1009 +0x40c fp=0xc820037fa8 sp=0xc820037e40
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1721 +0x1 fp=0xc820037fb0 sp=0xc820037fa8
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:686 +0xc9d

goroutine 83 [mark worker (idle)]:
runtime.gopark(0x104e818, 0xc82009b200, 0xf02300, 0x12, 0xc820124a14, 0x0)
	/usr/local/go/src/runtime/proc.go:185 +0x163 fp=0xc820024f50 sp=0xc820024f28
runtime.gcBgMarkWorker(0xc82001b500)
	/usr/local/go/src/runtime/mgc.go:1289 +0xf7 fp=0xc820024fb8 sp=0xc820024f50
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1721 +0x1 fp=0xc820024fc0 sp=0xc820024fb8
created by runtime.gcBgMarkStartWorkers
	/usr/local/go/src/runtime/mgc.go:1239 +0x93

goroutine 35 [select]:
runtime.gopark(0x104e9b8, 0xc820038ed8, 0xe70eb8, 0x6, 0x18, 0x2)
	/usr/local/go/src/runtime/proc.go:185 +0x163 fp=0xc820038b98 sp=0xc820038b70
runtime.selectgoImpl(0xc820038ed8, 0x0, 0x18)
	/usr/local/go/src/runtime/select.go:392 +0xa64 fp=0xc820038d40 sp=0xc820038b98
runtime.selectgo(0xc820038ed8)
	/usr/local/go/src/runtime/select.go:212 +0x12 fp=0xc820038d60 sp=0xc820038d40
net/http.(*persistConn).readLoop(0xc8201d60b0)
	/usr/local/go/src/net/http/transport.go:976 +0xac7 fp=0xc820038fa8 sp=0xc820038d60
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1721 +0x1 fp=0xc820038fb0 sp=0xc820038fa8
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:685 +0xc78

goroutine 36 [select]:
runtime.gopark(0x104e9b8, 0xc8201f6f10, 0xe70eb8, 0x6, 0x18, 0x2)
	/usr/local/go/src/runtime/proc.go:185 +0x163 fp=0xc8201f6c78 sp=0xc8201f6c50
runtime.selectgoImpl(0xc8201f6f10, 0x0, 0x18)
	/usr/local/go/src/runtime/select.go:392 +0xa64 fp=0xc8201f6e20 sp=0xc8201f6c78
runtime.selectgo(0xc8201f6f10)
	/usr/local/go/src/runtime/select.go:212 +0x12 fp=0xc8201f6e40 sp=0xc8201f6e20
net/http.(*persistConn).writeLoop(0xc8201d60b0)
	/usr/local/go/src/net/http/transport.go:1009 +0x40c fp=0xc8201f6fa8 sp=0xc8201f6e40
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1721 +0x1 fp=0xc8201f6fb0 sp=0xc8201f6fa8
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:686 +0xc9d

goroutine 52 [IO wait]:
runtime.gopark(0x104e940, 0x7fab15ef2448, 0xe69a28, 0x7, 0x4fc71b, 0x5)
	/usr/local/go/src/runtime/proc.go:185 +0x163 fp=0xc8202d2a08 sp=0xc8202d29e0
runtime.netpollblock(0x7fab15ef2420, 0xffffff0000000072, 0x0)
	/usr/local/go/src/runtime/netpoll.go:338 +0x15e fp=0xc8202d2a50 sp=0xc8202d2a08
net.runtime_pollWait(0x7fab15ef2420, 0x72, 0xc82000e160)
	/usr/local/go/src/runtime/netpoll.go:157 +0x60 fp=0xc8202d2a70 sp=0xc8202d2a50
net.(*pollDesc).Wait(0xc8202d4450, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x3a fp=0xc8202d2a90 sp=0xc8202d2a70
net.(*pollDesc).WaitRead(0xc8202d4450, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x36 fp=0xc8202d2ab8 sp=0xc8202d2a90
net.(*netFD).Read(0xc8202d43f0, 0xc820390000, 0x1000, 0x1000, 0x0, 0x7fab15eed050, 0xc82000e160)
	/usr/local/go/src/net/fd_unix.go:232 +0x23a fp=0xc8202d2b88 sp=0xc8202d2ab8
net.(*conn).Read(0xc8200fe1b0, 0xc820390000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:172 +0xe4 fp=0xc8202d2bf0 sp=0xc8202d2b88
net/http.noteEOFReader.Read(0x7fab15ef2be0, 0xc8200fe1b0, 0xc8202fe268, 0xc820390000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1370 +0x67 fp=0xc8202d2c40 sp=0xc8202d2bf0
net/http.(*noteEOFReader).Read(0xc82034aba0, 0xc820390000, 0x1000, 0x1000, 0xc8204f20b0, 0x0, 0x0)
	:126 +0xd0 fp=0xc8202d2c90 sp=0xc8202d2c40
bufio.(*Reader).fill(0xc8202c4660)
	/usr/local/go/src/bufio/bufio.go:97 +0x1e9 fp=0xc8202d2d50 sp=0xc8202d2c90
bufio.(*Reader).Peek(0xc8202c4660, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xcc fp=0xc8202d2d60 sp=0xc8202d2d50
net/http.(*persistConn).readLoop(0xc8202fe210)
	/usr/local/go/src/net/http/transport.go:876 +0xf7 fp=0xc8202d2fa8 sp=0xc8202d2d60
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1721 +0x1 fp=0xc8202d2fb0 sp=0xc8202d2fa8
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:685 +0xc78

goroutine 43 [chan receive]:
runtime.gopark(0x104e980, 0xc8201252b8, 0xe80770, 0xc, 0x17, 0x3)
	/usr/local/go/src/runtime/proc.go:185 +0x163 fp=0xc8201ee538 sp=0xc8201ee510
runtime.goparkunlock(0xc8201252b8, 0xe80770, 0xc, 0x17, 0x3)
	/usr/local/go/src/runtime/proc.go:191 +0x54 fp=0xc8201ee570 sp=0xc8201ee538
runtime.chanrecv(0xb38dc0, 0xc820125260, 0x0, 0xc8201ee601, 0xc8201e0000)
	/usr/local/go/src/runtime/chan.go:505 +0x901 fp=0xc8201ee630 sp=0xc8201ee570
runtime.chanrecv1(0xb38dc0, 0xc820125260, 0x0)
	/usr/local/go/src/runtime/chan.go:349 +0x2b fp=0xc8201ee660 sp=0xc8201ee630
github.com/couchbase/indexing/secondary/queryport/client.(*metadataClient).logstats(0xc8201b60f0)
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/meta_client.go:412 +0x99 fp=0xc8201ee7b8 sp=0xc8201ee660
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1721 +0x1 fp=0xc8201ee7c0 sp=0xc8201ee7b8
created by github.com/couchbase/indexing/secondary/queryport/client.newMetaBridgeClient
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/meta_client.go:68 +0x700

goroutine 53 [select]:
runtime.gopark(0x104e9b8, 0xc8202d3f10, 0xe70eb8, 0x6, 0xc82030d018, 0x2)
	/usr/local/go/src/runtime/proc.go:185 +0x163 fp=0xc8202d3c78 sp=0xc8202d3c50
runtime.selectgoImpl(0xc8202d3f10, 0x0, 0x18)
	/usr/local/go/src/runtime/select.go:392 +0xa64 fp=0xc8202d3e20 sp=0xc8202d3c78
runtime.selectgo(0xc8202d3f10)
	/usr/local/go/src/runtime/select.go:212 +0x12 fp=0xc8202d3e40 sp=0xc8202d3e20
net/http.(*persistConn).writeLoop(0xc8202fe210)
	/usr/local/go/src/net/http/transport.go:1009 +0x40c fp=0xc8202d3fa8 sp=0xc8202d3e40
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1721 +0x1 fp=0xc8202d3fb0 sp=0xc8202d3fa8
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:686 +0xc9d

goroutine 50 [select]:
runtime.gopark(0x104e9b8, 0xc8201f2ed8, 0xe70eb8, 0x6, 0x18, 0x2)
	/usr/local/go/src/runtime/proc.go:185 +0x163 fp=0xc8201f2b98 sp=0xc8201f2b70
runtime.selectgoImpl(0xc8201f2ed8, 0x0, 0x18)
	/usr/local/go/src/runtime/select.go:392 +0xa64 fp=0xc8201f2d40 sp=0xc8201f2b98
runtime.selectgo(0xc8201f2ed8)
	/usr/local/go/src/runtime/select.go:212 +0x12 fp=0xc8201f2d60 sp=0xc8201f2d40
net/http.(*persistConn).readLoop(0xc8201d62c0)
	/usr/local/go/src/net/http/transport.go:976 +0xac7 fp=0xc8201f2fa8 sp=0xc8201f2d60
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1721 +0x1 fp=0xc8201f2fb0 sp=0xc8201f2fa8
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:685 +0xc78

goroutine 51 [select]:
runtime.gopark(0x104e9b8, 0xc8202d0f10, 0xe70eb8, 0x6, 0x18, 0x2)
	/usr/local/go/src/runtime/proc.go:185 +0x163 fp=0xc8202d0c78 sp=0xc8202d0c50
runtime.selectgoImpl(0xc8202d0f10, 0x0, 0x18)
	/usr/local/go/src/runtime/select.go:392 +0xa64 fp=0xc8202d0e20 sp=0xc8202d0c78
runtime.selectgo(0xc8202d0f10)
	/usr/local/go/src/runtime/select.go:212 +0x12 fp=0xc8202d0e40 sp=0xc8202d0e20
net/http.(*persistConn).writeLoop(0xc8201d62c0)
	/usr/local/go/src/net/http/transport.go:1009 +0x40c fp=0xc8202d0fa8 sp=0xc8202d0e40
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1721 +0x1 fp=0xc8202d0fb0 sp=0xc8202d0fa8
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:686 +0xc9d

goroutine 7 [IO wait]:
runtime.gopark(0x104e940, 0x7fab15ef28c8, 0xe69a28, 0x7, 0x4fc71b, 0x5)
	/usr/local/go/src/runtime/proc.go:185 +0x163 fp=0xc8200492e8 sp=0xc8200492c0
runtime.netpollblock(0x7fab15ef28a0, 0xffffff0000000072, 0x0)
	/usr/local/go/src/runtime/netpoll.go:338 +0x15e fp=0xc820049330 sp=0xc8200492e8
net.runtime_pollWait(0x7fab15ef28a0, 0x72, 0xc82000e160)
	/usr/local/go/src/runtime/netpoll.go:157 +0x60 fp=0xc820049350 sp=0xc820049330
net.(*pollDesc).Wait(0xc8201b43e0, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x3a fp=0xc820049370 sp=0xc820049350
net.(*pollDesc).WaitRead(0xc8201b43e0, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x36 fp=0xc820049398 sp=0xc820049370
net.(*netFD).Read(0xc8201b4380, 0xc8201db000, 0x1000, 0x1000, 0x0, 0x7fab15eed050, 0xc82000e160)
	/usr/local/go/src/net/fd_unix.go:232 +0x23a fp=0xc820049468 sp=0xc820049398
net.(*conn).Read(0xc8201bc038, 0xc8201db000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:172 +0xe4 fp=0xc8200494d0 sp=0xc820049468
net/http.noteEOFReader.Read(0x7fab15ef2be0, 0xc8201bc038, 0xc8201d6108, 0xc8201db000, 0x1000, 0x1000, 0xc82000d9e8, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1370 +0x67 fp=0xc820049520 sp=0xc8200494d0
net/http.(*noteEOFReader).Read(0xc8201b0c00, 0xc8201db000, 0x1000, 0x1000, 0xc, 0x0, 0x0)
	:126 +0xd0 fp=0xc820049570 sp=0xc820049520
bufio.(*Reader).fill(0xc8201ac480)
	/usr/local/go/src/bufio/bufio.go:97 +0x1e9 fp=0xc820049630 sp=0xc820049570
bufio.(*Reader).ReadSlice(0xc8201ac480, 0xa, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:328 +0x21a fp=0xc820049678 sp=0xc820049630
net/http/internal.readLine(0xc8201ac480, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/internal/chunked.go:110 +0x4b fp=0xc8200496b8 sp=0xc820049678
net/http/internal.(*chunkedReader).beginChunk(0xc820202990)
	/usr/local/go/src/net/http/internal/chunked.go:47 +0x39 fp=0xc820049718 sp=0xc8200496b8
net/http/internal.(*chunkedReader).Read(0xc820202990, 0xc82030a000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/internal/chunked.go:77 +0xb7 fp=0xc8200497e0 sp=0xc820049718
net/http.(*body).readLocked(0xc820107300, 0xc82030a000, 0x1000, 0x1000, 0xc800000000, 0x0, 0x0)
	/usr/local/go/src/net/http/transfer.go:632 +0x98 fp=0xc820049830 sp=0xc8200497e0
net/http.(*body).Read(0xc820107300, 0xc82030a000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transfer.go:624 +0x110 fp=0xc820049880 sp=0xc820049830
net/http.(*bodyEOFSignal).Read(0xc820107340, 0xc82030a000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1296 +0x26a fp=0xc820049920 sp=0xc820049880
bufio.(*Reader).fill(0xc820049d60)
	/usr/local/go/src/bufio/bufio.go:97 +0x1e9 fp=0xc8200499e0 sp=0xc820049920
bufio.(*Reader).ReadSlice(0xc820049d60, 0xc82030a10a, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:328 +0x21a fp=0xc820049a28 sp=0xc8200499e0
bufio.(*Reader).ReadBytes(0xc820049d60, 0xc82000d90a, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:406 +0xa9 fp=0xc820049b48 sp=0xc820049a28
github.com/couchbase/indexing/secondary/dcp.(*Client).runObserveStreamingEndpoint(0xc8200c3430, 0xc8202c2210, 0x24, 0x104a9a0, 0xc82000d800, 0x0, 0x0, 0x0)
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/dcp/pools.go:366 +0x806 fp=0xc820049e90 sp=0xc820049b48
github.com/couchbase/indexing/secondary/dcp.(*Client).RunObserveNodeServices(0xc8200c3430, 0xe6d388, 0x7, 0xc82000d800, 0x0, 0x0, 0x0)
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/dcp/pools.go:318 +0xc3 fp=0xc820049ef0 sp=0xc820049e90
github.com/couchbase/indexing/secondary/common.(*serviceNotifierInstance).RunServicesObserver(0xc8200c33f0)
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/common/services_notifier.go:87 +0x9a fp=0xc820049f98 sp=0xc820049ef0
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1721 +0x1 fp=0xc820049fa0 sp=0xc820049f98
created by github.com/couchbase/indexing/secondary/common.NewServicesChangeNotifier
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/common/services_notifier.go:157 +0x560

goroutine 84 [mark worker (idle)]:
runtime.gopark(0x104e818, 0xc8200e8480, 0xf02300, 0x12, 0xc820126314, 0x0)
	/usr/local/go/src/runtime/proc.go:185 +0x163 fp=0xc820024750 sp=0xc820024728
runtime.gcBgMarkWorker(0xc82001ca00)
	/usr/local/go/src/runtime/mgc.go:1289 +0xf7 fp=0xc8200247b8 sp=0xc820024750
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1721 +0x1 fp=0xc8200247c0 sp=0xc8200247b8
created by runtime.gcBgMarkStartWorkers
	/usr/local/go/src/runtime/mgc.go:1239 +0x93

goroutine 9 [select]:
runtime.gopark(0x104e9b8, 0xc82003bed8, 0xe70eb8, 0x6, 0x18, 0x2)
	/usr/local/go/src/runtime/proc.go:185 +0x163 fp=0xc82003bb98 sp=0xc82003bb70
runtime.selectgoImpl(0xc82003bed8, 0x0, 0x18)
	/usr/local/go/src/runtime/select.go:392 +0xa64 fp=0xc82003bd40 sp=0xc82003bb98
runtime.selectgo(0xc82003bed8)
	/usr/local/go/src/runtime/select.go:212 +0x12 fp=0xc82003bd60 sp=0xc82003bd40
net/http.(*persistConn).readLoop(0xc8200c34a0)
	/usr/local/go/src/net/http/transport.go:976 +0xac7 fp=0xc82003bfa8 sp=0xc82003bd60
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1721 +0x1 fp=0xc82003bfb0 sp=0xc82003bfa8
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:685 +0xc78

goroutine 10 [select]:
runtime.gopark(0x104e9b8, 0xc8201f3f10, 0xe70eb8, 0x6, 0x18, 0x2)
	/usr/local/go/src/runtime/proc.go:185 +0x163 fp=0xc8201f3c78 sp=0xc8201f3c50
runtime.selectgoImpl(0xc8201f3f10, 0x0, 0x18)
	/usr/local/go/src/runtime/select.go:392 +0xa64 fp=0xc8201f3e20 sp=0xc8201f3c78
runtime.selectgo(0xc8201f3f10)
	/usr/local/go/src/runtime/select.go:212 +0x12 fp=0xc8201f3e40 sp=0xc8201f3e20
net/http.(*persistConn).writeLoop(0xc8200c34a0)
	/usr/local/go/src/net/http/transport.go:1009 +0x40c fp=0xc8201f3fa8 sp=0xc8201f3e40
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1721 +0x1 fp=0xc8201f3fb0 sp=0xc8201f3fa8
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:686 +0xc9d

goroutine 12 [IO wait]:
runtime.gopark(0x104e940, 0x7fab15ef25c8, 0xe69a28, 0x7, 0x4fc71b, 0x5)
	/usr/local/go/src/runtime/proc.go:185 +0x163 fp=0xc82003ca08 sp=0xc82003c9e0
runtime.netpollblock(0x7fab15ef25a0, 0xffffff0000000072, 0x0)
	/usr/local/go/src/runtime/netpoll.go:338 +0x15e fp=0xc82003ca50 sp=0xc82003ca08
net.runtime_pollWait(0x7fab15ef25a0, 0x72, 0xc82000e160)
	/usr/local/go/src/runtime/netpoll.go:157 +0x60 fp=0xc82003ca70 sp=0xc82003ca50
net.(*pollDesc).Wait(0xc820056920, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x3a fp=0xc82003ca90 sp=0xc82003ca70
net.(*pollDesc).WaitRead(0xc820056920, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x36 fp=0xc82003cab8 sp=0xc82003ca90
net.(*netFD).Read(0xc8200568c0, 0xc82030d000, 0x1000, 0x1000, 0x0, 0x7fab15eed050, 0xc82000e160)
	/usr/local/go/src/net/fd_unix.go:232 +0x23a fp=0xc82003cb88 sp=0xc82003cab8
net.(*conn).Read(0xc82002a0d8, 0xc82030d000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:172 +0xe4 fp=0xc82003cbf0 sp=0xc82003cb88
net/http.noteEOFReader.Read(0x7fab15ef2be0, 0xc82002a0d8, 0xc8200c35a8, 0xc82030d000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1370 +0x67 fp=0xc82003cc40 sp=0xc82003cbf0
net/http.(*noteEOFReader).Read(0xc8203283c0, 0xc82030d000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	:126 +0xd0 fp=0xc82003cc90 sp=0xc82003cc40
bufio.(*Reader).fill(0xc820014de0)
	/usr/local/go/src/bufio/bufio.go:97 +0x1e9 fp=0xc82003cd50 sp=0xc82003cc90
bufio.(*Reader).Peek(0xc820014de0, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xcc fp=0xc82003cd60 sp=0xc82003cd50
net/http.(*persistConn).readLoop(0xc8200c3550)
	/usr/local/go/src/net/http/transport.go:876 +0xf7 fp=0xc82003cfa8 sp=0xc82003cd60
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1721 +0x1 fp=0xc82003cfb0 sp=0xc82003cfa8
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:685 +0xc78

goroutine 13 [select]:
runtime.gopark(0x104e9b8, 0xc82003df10, 0xe70eb8, 0x6, 0x18, 0x2)
	/usr/local/go/src/runtime/proc.go:185 +0x163 fp=0xc82003dc78 sp=0xc82003dc50
runtime.selectgoImpl(0xc82003df10, 0x0, 0x18)
	/usr/local/go/src/runtime/select.go:392 +0xa64 fp=0xc82003de20 sp=0xc82003dc78
runtime.selectgo(0xc82003df10)
	/usr/local/go/src/runtime/select.go:212 +0x12 fp=0xc82003de40 sp=0xc82003de20
net/http.(*persistConn).writeLoop(0xc8200c3550)
	/usr/local/go/src/net/http/transport.go:1009 +0x40c fp=0xc82003dfa8 sp=0xc82003de40
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1721 +0x1 fp=0xc82003dfb0 sp=0xc82003dfa8
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:686 +0xc9d

goroutine 30 [select]:
runtime.gopark(0x104e9b8, 0xc8201f8f10, 0xe70eb8, 0x6, 0xc8201db018, 0x2)
	/usr/local/go/src/runtime/proc.go:185 +0x163 fp=0xc8201f8c78 sp=0xc8201f8c50
runtime.selectgoImpl(0xc8201f8f10, 0x0, 0x18)
	/usr/local/go/src/runtime/select.go:392 +0xa64 fp=0xc8201f8e20 sp=0xc8201f8c78
runtime.selectgo(0xc8201f8f10)
	/usr/local/go/src/runtime/select.go:212 +0x12 fp=0xc8201f8e40 sp=0xc8201f8e20
net/http.(*persistConn).writeLoop(0xc8201104d0)
	/usr/local/go/src/net/http/transport.go:1009 +0x40c fp=0xc8201f8fa8 sp=0xc8201f8e40
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1721 +0x1 fp=0xc8201f8fb0 sp=0xc8201f8fa8
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:686 +0xc9d

goroutine 33 [Concurrent GC wait]:
runtime.gopark(0x104e980, 0x160a2e0, 0xeed800, 0x12, 0xc820027d14, 0x1)
	/usr/local/go/src/runtime/proc.go:185 +0x163 fp=0xc820027f58 sp=0xc820027f30
runtime.goparkunlock(0x160a2e0, 0xeed800, 0x12, 0x10014, 0x1)
	/usr/local/go/src/runtime/proc.go:191 +0x54 fp=0xc820027f90 sp=0xc820027f58
runtime.backgroundgc()
	/usr/local/go/src/runtime/mgc.go:900 +0x8b fp=0xc820027fc0 sp=0xc820027f90
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1721 +0x1 fp=0xc820027fc8 sp=0xc820027fc0
created by runtime.startGC
	/usr/local/go/src/runtime/mgc.go:870 +0x186

goroutine 59 [runnable]:
fmt.(*fmt).pad(0xc8201c4538, 0xc8201c4576, 0x3, 0x3)
	/usr/local/go/src/fmt/format.go:112 fp=0xc8201f93a8 sp=0xc8201f93a0
fmt.(*fmt).integer(0xc8201c4538, 0xa5, 0xa, 0xb50501, 0xeecc60, 0x10)
	/usr/local/go/src/fmt/format.go:301 +0x4ff fp=0xc8201f9490 sp=0xc8201f93a8
fmt.(*pp).fmtInt64(0xc8201c44e0, 0xa5, 0xc800000064)
	/usr/local/go/src/fmt/print.go:368 +0x15a fp=0xc8201f94c8 sp=0xc8201f9490
fmt.(*pp).printArg(0xc8201c44e0, 0xb505a0, 0xc8202171b0, 0x64, 0x0, 0x0)
	/usr/local/go/src/fmt/print.go:775 +0xf32 fp=0xc8201f9650 sp=0xc8201f94c8
fmt.(*pp).doPrintf(0xc8201c44e0, 0xe72cf0, 0xd, 0xc8201f9bc8, 0x3, 0x3)
	/usr/local/go/src/fmt/print.go:1219 +0x1dd8 fp=0xc8201f99d8 sp=0xc8201f9650
fmt.Fprintf(0x7fab1408be40, 0xc820114540, 0xe72cf0, 0xd, 0xc8201f9bc8, 0x3, 0x3, 0x4, 0x0, 0x0)
	/usr/local/go/src/fmt/print.go:188 +0x74 fp=0xc8201f9a20 sp=0xc8201f99d8
runtime/debug.stack(0x0, 0x0, 0x0)
	/usr/local/go/src/runtime/debug/stack.go:52 +0x356 fp=0xc8201f9c00 sp=0xc8201f9a20
runtime/debug.Stack(0x0, 0x0, 0x0)
	/usr/local/go/src/runtime/debug/stack.go:36 +0x29 fp=0xc8201f9c20 sp=0xc8201f9c00
github.com/couchbase/indexing/secondary/logging.(*destination).StackTrace(0x1609180, 0x0, 0x0)
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/logging/logging.go:165 +0x24 fp=0xc8201f9c78 sp=0xc8201f9c20
github.com/couchbase/gometa/common.(*PeerPipe).Close(0xc820260b70, 0xc8201f9d00)
	/opt/build/goproj/src/github.com/couchbase/gometa/common/peerPipe.go:96 +0x295 fp=0xc8201f9d08 sp=0xc8201f9c78
github.com/couchbase/gometa/protocol.runOnce.func2()
	/opt/build/goproj/src/github.com/couchbase/gometa/protocol/watcherServer.go:168 +0x20 fp=0xc8201f9d20 sp=0xc8201f9d08
github.com/couchbase/gometa/common.SafeRun(0xef1ea0, 0x17, 0xc8201f9dd8)
	/opt/build/goproj/src/github.com/couchbase/gometa/common/util.go:36 +0x4e fp=0xc8201f9d48 sp=0xc8201f9d20
github.com/couchbase/gometa/protocol.runOnce(0xc8200f8100, 0xe, 0x7fab14043388, 0xc8202ea240, 0x7fab140433c0, 0xc8202ea240, 0x7fab14043300, 0x1631058, 0xc820340230, 0xc8201ac5a0, ...)
	/opt/build/goproj/src/github.com/couchbase/gometa/protocol/watcherServer.go:186 +0x4cd fp=0xc8201f9e28 sp=0xc8201f9d48
github.com/couchbase/gometa/protocol.RunWatcherServerWithRequest(0xc8200f8100, 0xe, 0x7fab14043388, 0xc8202ea240, 0x7fab140433c0, 0xc8202ea240, 0x7fab14043300, 0x1631058, 0xc820340230, 0xc8201ac5a0, ...)
	/opt/build/goproj/src/github.com/couchbase/gometa/protocol/watcherServer.go:49 +0x146 fp=0xc8201f9f50 sp=0xc8201f9e28
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1721 +0x1 fp=0xc8201f9f58 sp=0xc8201f9f50
created by github.com/couchbase/indexing/secondary/manager/client.(*MetadataProvider).startWatcher
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/manager/client/metadata_provider.go:672 +0x171

goroutine 82 [mark worker (idle)]:
runtime.gopark(0x104e818, 0xc82009bb00, 0xf02300, 0x12, 0xc820340c14, 0x0)
	/usr/local/go/src/runtime/proc.go:185 +0x163 fp=0xc8201ec750 sp=0xc8201ec728
runtime.gcBgMarkWorker(0xc82001a000)
	/usr/local/go/src/runtime/mgc.go:1289 +0xf7 fp=0xc8201ec7b8 sp=0xc8201ec750
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1721 +0x1 fp=0xc8201ec7c0 sp=0xc8201ec7b8
created by runtime.gcBgMarkStartWorkers
	/usr/local/go/src/runtime/mgc.go:1239 +0x93

goroutine 55 [chan receive]:
runtime.gopark(0x104e980, 0xc8204e2178, 0xe80770, 0xc, 0x17, 0x3)
	/usr/local/go/src/runtime/proc.go:185 +0x163 fp=0xc8201ed538 sp=0xc8201ed510
runtime.goparkunlock(0xc8204e2178, 0xe80770, 0xc, 0x17, 0x3)
	/usr/local/go/src/runtime/proc.go:191 +0x54 fp=0xc8201ed570 sp=0xc8201ed538
runtime.chanrecv(0xb38dc0, 0xc8204e2120, 0x0, 0xc8201ed601, 0xc8201e0000)
	/usr/local/go/src/runtime/chan.go:505 +0x901 fp=0xc8201ed630 sp=0xc8201ed570
runtime.chanrecv1(0xb38dc0, 0xc8204e2120, 0x0)
	/usr/local/go/src/runtime/chan.go:349 +0x2b fp=0xc8201ed660 sp=0xc8201ed630
github.com/couchbase/indexing/secondary/queryport/client.(*metadataClient).logstats(0xc82000aeb0)
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/meta_client.go:412 +0x99 fp=0xc8201ed7b8 sp=0xc8201ed660
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1721 +0x1 fp=0xc8201ed7c0 sp=0xc8201ed7b8
created by github.com/couchbase/indexing/secondary/queryport/client.newMetaBridgeClient
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/meta_client.go:68 +0x700

goroutine 60 [chan receive]:
runtime.gopark(0x104e980, 0xc8201ac6b8, 0xe80770, 0xc, 0x17, 0x3)
	/usr/local/go/src/runtime/proc.go:185 +0x163 fp=0xc8201f5d88 sp=0xc8201f5d60
runtime.goparkunlock(0xc8201ac6b8, 0xe80770, 0xc, 0x17, 0x3)
	/usr/local/go/src/runtime/proc.go:191 +0x54 fp=0xc8201f5dc0 sp=0xc8201f5d88
runtime.chanrecv(0xb394e0, 0xc8201ac660, 0xc8201f5f60, 0x1, 0x0)
	/usr/local/go/src/runtime/chan.go:505 +0x901 fp=0xc8201f5e80 sp=0xc8201f5dc0
runtime.chanrecv2(0xb394e0, 0xc8201ac660, 0xc8201f5f60, 0x35)
	/usr/local/go/src/runtime/chan.go:354 +0x2b fp=0xc8201f5eb0 sp=0xc8201f5e80
github.com/couchbase/gometa/common.(*PeerPipe).doSend(0xc820260b70)
	/opt/build/goproj/src/github.com/couchbase/gometa/common/peerPipe.go:153 +0x8d fp=0xc8201f5fa8 sp=0xc8201f5eb0
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1721 +0x1 fp=0xc8201f5fb0 sp=0xc8201f5fa8
created by github.com/couchbase/gometa/common.NewPeerPipe
	/opt/build/goproj/src/github.com/couchbase/gometa/common/peerPipe.go:59 +0x120

goroutine 85 [mark worker (idle)]:
runtime.gopark(0x104e818, 0xc82009ad80, 0xf02300, 0x12, 0x14, 0x0)
	/usr/local/go/src/runtime/proc.go:185 +0x163 fp=0xc820025750 sp=0xc820025728
runtime.gcBgMarkWorker(0xc82001e000)
	/usr/local/go/src/runtime/mgc.go:1289 +0xf7 fp=0xc8200257b8 sp=0xc820025750
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1721 +0x1 fp=0xc8200257c0 sp=0xc8200257b8
created by runtime.gcBgMarkStartWorkers
	/usr/local/go/src/runtime/mgc.go:1239 +0x93

goroutine 61 [runnable]:
syscall.Syscall(0x0, 0x5, 0xc820216ec0, 0x8, 0x0, 0x8, 0x0)
	/usr/local/go/src/syscall/asm_linux_amd64.s:18 +0x5 fp=0xc8204fbc10 sp=0xc8204fbc08
syscall.read(0x5, 0xc820216ec0, 0x8, 0x8, 0x72, 0x0, 0x0)
	/usr/local/go/src/syscall/zsyscall_linux_amd64.go:783 +0x5f fp=0xc8204fbc50 sp=0xc8204fbc10
syscall.Read(0x5, 0xc820216ec0, 0x8, 0x8, 0xffffffffffffff01, 0x0, 0x0)
	/usr/local/go/src/syscall/syscall_unix.go:160 +0x4d fp=0xc8204fbc90 sp=0xc8204fbc50
net.(*netFD).Read(0xc8203403f0, 0xc820216ec0, 0x8, 0x8, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/fd_unix.go:228 +0x18b fp=0xc8204fbd60 sp=0xc8204fbc90
net.(*conn).Read(0xc8200fe018, 0xc820216ec0, 0x8, 0x8, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:172 +0xe4 fp=0xc8204fbdc8 sp=0xc8204fbd60
github.com/couchbase/gometa/common.(*PeerPipe).readBytes(0xc820260b70, 0x8, 0x0, 0x0, 0x0, 0x0, 0x0)
	/opt/build/goproj/src/github.com/couchbase/gometa/common/peerPipe.go:256 +0xef fp=0xc8204fbeb0 sp=0xc8204fbdc8
github.com/couchbase/gometa/common.(*PeerPipe).doReceive(0xc820260b70)
	/opt/build/goproj/src/github.com/couchbase/gometa/common/peerPipe.go:201 +0x67 fp=0xc8204fbfa8 sp=0xc8204fbeb0
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1721 +0x1 fp=0xc8204fbfb0 sp=0xc8204fbfa8
created by github.com/couchbase/gometa/common.NewPeerPipe
	/opt/build/goproj/src/github.com/couchbase/gometa/common/peerPipe.go:60 +0x142

goroutine 64 [runnable]:
runtime.gopark(0x104e9b8, 0xc8201edf20, 0xe70eb8, 0x6, 0x18, 0x2)
	/usr/local/go/src/runtime/proc.go:185 +0x163 fp=0xc8201edc98 sp=0xc8201edc70
runtime.selectgoImpl(0xc8201edf20, 0x0, 0x18)
	/usr/local/go/src/runtime/select.go:392 +0xa64 fp=0xc8201ede40 sp=0xc8201edc98
runtime.selectgo(0xc8201edf20)
	/usr/local/go/src/runtime/select.go:212 +0x12 fp=0xc8201ede60 sp=0xc8201ede40
github.com/couchbase/gometa/protocol.(*Follower).startListener(0xc820340a10)
	/opt/build/goproj/src/github.com/couchbase/gometa/protocol/follower.go:149 +0x36c fp=0xc8201edfb8 sp=0xc8201ede60
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1721 +0x1 fp=0xc8201edfc0 sp=0xc8201edfb8
created by github.com/couchbase/gometa/protocol.(*Follower).Start
	/opt/build/goproj/src/github.com/couchbase/gometa/protocol/follower.go:76 +0x35

goroutine 65 [runnable]:
runtime.gopark(0x104e9b8, 0xc8201eb720, 0xe70eb8, 0x6, 0xc81ffd7e18, 0x2)
	/usr/local/go/src/runtime/proc.go:185 +0x163 fp=0xc8201eb520 sp=0xc8201eb4f8
runtime.selectgoImpl(0xc8201eb720, 0x0, 0x18)
	/usr/local/go/src/runtime/select.go:392 +0xa64 fp=0xc8201eb6c8 sp=0xc8201eb520
runtime.selectgo(0xc8201eb720)
	/usr/local/go/src/runtime/select.go:212 +0x12 fp=0xc8201eb6e8 sp=0xc8201eb6c8
github.com/couchbase/indexing/secondary/manager/client.(*watcher).timeoutChecker(0xc8202ea240)
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/manager/client/metadata_provider.go:1326 +0x131 fp=0xc8201eb7b8 sp=0xc8201eb6e8
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1721 +0x1 fp=0xc8201eb7c0 sp=0xc8201eb7b8
created by github.com/couchbase/indexing/secondary/manager/client.(*watcher).startTimer
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/manager/client/metadata_provider.go:1317 +0x6b

goroutine 86 [runnable]:
runtime.gopark(0x104e9b8, 0xc8201f4f10, 0xe70eb8, 0x6, 0xc8204f1d18, 0x2)
	/usr/local/go/src/runtime/proc.go:185 +0x163 fp=0xc8201f4c70 sp=0xc8201f4c48
runtime.selectgoImpl(0xc8201f4f10, 0x0, 0x18)
	/usr/local/go/src/runtime/select.go:392 +0xa64 fp=0xc8201f4e18 sp=0xc8201f4c70
runtime.selectgo(0xc8201f4f10)
	/usr/local/go/src/runtime/select.go:212 +0x12 fp=0xc8201f4e38 sp=0xc8201f4e18
github.com/couchbase/indexing/secondary/queryport/client.(*metadataClient).watchClusterChanges(0xc8201b6d70)
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/meta_client.go:774 +0x52d fp=0xc8201f4fa8 sp=0xc8201f4e38
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1721 +0x1 fp=0xc8201f4fb0 sp=0xc8201f4fa8
created by github.com/couchbase/indexing/secondary/queryport/client.newMetaBridgeClient
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/meta_client.go:67 +0x6de

goroutine 87 [chan receive]:
runtime.gopark(0x104e980, 0xc8201ac8f8, 0xe80770, 0xc, 0x17, 0x3)
	/usr/local/go/src/runtime/proc.go:185 +0x163 fp=0xc820023538 sp=0xc820023510
runtime.goparkunlock(0xc8201ac8f8, 0xe80770, 0xc, 0x17, 0x3)
	/usr/local/go/src/runtime/proc.go:191 +0x54 fp=0xc820023570 sp=0xc820023538
runtime.chanrecv(0xb38dc0, 0xc8201ac8a0, 0x0, 0xc820023601, 0xc820020000)
	/usr/local/go/src/runtime/chan.go:505 +0x901 fp=0xc820023630 sp=0xc820023570
runtime.chanrecv1(0xb38dc0, 0xc8201ac8a0, 0x0)
	/usr/local/go/src/runtime/chan.go:349 +0x2b fp=0xc820023660 sp=0xc820023630
github.com/couchbase/indexing/secondary/queryport/client.(*metadataClient).logstats(0xc8201b6d70)
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/meta_client.go:412 +0x99 fp=0xc8200237b8 sp=0xc820023660
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1721 +0x1 fp=0xc8200237c0 sp=0xc8200237b8
created by github.com/couchbase/indexing/secondary/queryport/client.newMetaBridgeClient
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/meta_client.go:68 +0x700
signal: aborted (core dumped)
FAIL	github.com/couchbase/indexing/secondary/tests/functionaltests	0.218s
2016/04/06 08:20:32 In init()
2016/04/06 08:20:32 Using forestdb for creating indexes
2016/04/06 08:20:32 Changing config key indexer.settings.storage_mode to value forestdb
2016/04/06 08:20:32 Type of settings value is string
2016/04/06 08:20:32 Type of config value that was updated is string
2016/04/06 08:20:32 After updation, Setting  is indexer.settings.storage_mode = forestdb
2016/04/06 08:20:32 Settings updated indexer.settings.storage_mode = forestdb
=== RUN   TestRangeWithConcurrentAddMuts
2016/04/06 08:20:37 In TestRangeWithConcurrentAddMuts()
2016/04/06 08:20:37 In DropAllSecondaryIndexes()
2016/04/06 08:20:38 Generating JSON docs
2016/04/06 08:20:38 Setting initial JSON docs in KV
2016/04/06 08:20:38 All indexers are active
2016/04/06 08:20:38 Creating a 2i
2016/04/06 08:20:40 Created the secondary index index_company. Waiting for it become active
2016/04/06 08:20:40 Index state is INDEX_STATE_ACTIVE
2016/04/06 08:20:40 Index is now active
2016/04/06 08:20:40 PERFSTAT CreateAndBuildIndex 2.2161 seconds
2016/04/06 08:20:40 In Range Scan
2016/04/06 08:20:40 CreateDocs:: Creating mutations
2016/04/06 08:20:40 PERFSTAT Range 0.0033 seconds
2016/04/06 08:20:40 Range Scan of 1000 user documents took 3.301549ms
2016/04/06 08:20:40 PERFSTAT Range 0.0073 seconds
2016/04/06 08:20:40 Range Scan of 1000 user documents took 7.365923ms
2016/04/06 08:20:40 PERFSTAT Range 0.0385 seconds
2016/04/06 08:20:40 Range Scan of 1000 user documents took 38.775182ms
2016/04/06 08:20:40 PERFSTAT Range 0.0039 seconds
2016/04/06 08:20:40 Range Scan of 1000 user documents took 3.929186ms
2016/04/06 08:20:40 PERFSTAT Range 0.0007 seconds
2016/04/06 08:20:40 Range Scan of 1000 user documents took 806.09µs
2016/04/06 08:20:40 PERFSTAT Range 0.0027 seconds
2016/04/06 08:20:40 Range Scan of 1000 user documents took 3.2882ms
2016/04/06 08:20:40 PERFSTAT Range 0.0056 seconds
2016/04/06 08:20:40 Range Scan of 1000 user documents took 5.589087ms
2016/04/06 08:20:40 PERFSTAT Range 0.0012 seconds
2016/04/06 08:20:40 Range Scan of 1000 user documents took 1.232136ms
2016/04/06 08:20:40 PERFSTAT Range 0.0015 seconds
2016/04/06 08:20:40 Range Scan of 1000 user documents took 1.518306ms
2016/04/06 08:20:40 PERFSTAT Range 0.0004 seconds
2016/04/06 08:20:40 Range Scan of 1000 user documents took 454.221µs
2016/04/06 08:20:40 PERFSTAT Range 0.0011 seconds
2016/04/06 08:20:40 Range Scan of 1000 user documents took 1.156515ms
2016/04/06 08:20:40 PERFSTAT Range 0.0009 seconds
2016/04/06 08:20:40 Range Scan of 1000 user documents took 970.878µs
2016/04/06 08:20:40 PERFSTAT Range 0.0011 seconds
2016/04/06 08:20:40 Range Scan of 1000 user documents took 1.111728ms
2016/04/06 08:20:40 PERFSTAT Range 0.0022 seconds
2016/04/06 08:20:40 Range Scan of 1000 user documents took 2.198075ms
2016/04/06 08:20:40 PERFSTAT Range 0.0447 seconds
2016/04/06 08:20:40 Range Scan of 1100 user documents took 45.912719ms
2016/04/06 08:20:40 PERFSTAT Range 0.0024 seconds
2016/04/06 08:20:40 Range Scan of 1100 user documents took 2.46234ms
2016/04/06 08:20:40 PERFSTAT Range 0.0004 seconds
2016/04/06 08:20:40 Range Scan of 1100 user documents took 455.689µs
2016/04/06 08:20:41 PERFSTAT Range 0.0092 seconds
2016/04/06 08:20:41 Range Scan of 1100 user documents took 9.261642ms
2016/04/06 08:20:41 PERFSTAT Range 0.0149 seconds
2016/04/06 08:20:41 Range Scan of 1100 user documents took 14.913832ms
2016/04/06 08:20:41 PERFSTAT Range 0.0012 seconds
2016/04/06 08:20:41 Range Scan of 1100 user documents took 6.513398ms
2016/04/06 08:20:41 PERFSTAT Range 0.0012 seconds
2016/04/06 08:20:41 Range Scan of 1100 user documents took 1.210835ms
2016/04/06 08:20:41 PERFSTAT Range 0.0005 seconds
2016/04/06 08:20:41 Range Scan of 1100 user documents took 627.811µs
2016/04/06 08:20:41 PERFSTAT Range 0.0125 seconds
2016/04/06 08:20:41 Range Scan of 1100 user documents took 12.57002ms
2016/04/06 08:20:41 PERFSTAT Range 0.0154 seconds
2016/04/06 08:20:41 Range Scan of 1100 user documents took 15.462467ms
2016/04/06 08:20:41 PERFSTAT Range 0.0006 seconds
2016/04/06 08:20:41 Range Scan of 1100 user documents took 656.472µs
2016/04/06 08:20:41 PERFSTAT Range 0.0391 seconds
2016/04/06 08:20:41 Range Scan of 1200 user documents took 39.18649ms
2016/04/06 08:20:41 PERFSTAT Range 0.0006 seconds
2016/04/06 08:20:41 Range Scan of 1200 user documents took 678.215µs
2016/04/06 08:20:41 PERFSTAT Range 0.0006 seconds
2016/04/06 08:20:41 Range Scan of 1200 user documents took 645.011µs
2016/04/06 08:20:41 PERFSTAT Range 0.1500 seconds
2016/04/06 08:20:41 Range Scan of 1200 user documents took 150.064804ms
2016/04/06 08:20:41 PERFSTAT Range 0.0293 seconds
2016/04/06 08:20:41 Range Scan of 1300 user documents took 29.331482ms
2016/04/06 08:20:41 PERFSTAT Range 0.0105 seconds
2016/04/06 08:20:41 Range Scan of 1300 user documents took 10.581144ms
2016/04/06 08:20:41 PERFSTAT Range 0.0006 seconds
2016/04/06 08:20:41 Range Scan of 1300 user documents took 636.398µs
2016/04/06 08:20:41 PERFSTAT Range 0.0011 seconds
2016/04/06 08:20:41 Range Scan of 1300 user documents took 1.130525ms
2016/04/06 08:20:41 PERFSTAT Range 0.0007 seconds
2016/04/06 08:20:41 Range Scan of 1300 user documents took 774.652µs
2016/04/06 08:20:41 PERFSTAT Range 0.0124 seconds
2016/04/06 08:20:41 Range Scan of 1300 user documents took 12.424038ms
2016/04/06 08:20:41 PERFSTAT Range 0.0380 seconds
2016/04/06 08:20:41 Range Scan of 1300 user documents took 38.196277ms
2016/04/06 08:20:41 PERFSTAT Range 0.0005 seconds
2016/04/06 08:20:41 Range Scan of 1300 user documents took 488.833µs
2016/04/06 08:20:41 PERFSTAT Range 0.0030 seconds
2016/04/06 08:20:41 Range Scan of 1300 user documents took 3.023478ms
2016/04/06 08:20:41 PERFSTAT Range 0.0044 seconds
2016/04/06 08:20:41 Range Scan of 1300 user documents took 4.387647ms
2016/04/06 08:20:41 PERFSTAT Range 0.0284 seconds
2016/04/06 08:20:41 Range Scan of 1300 user documents took 28.402254ms
2016/04/06 08:20:41 PERFSTAT Range 0.0120 seconds
2016/04/06 08:20:41 Range Scan of 1300 user documents took 12.009568ms
2016/04/06 08:20:41 PERFSTAT Range 0.0367 seconds
2016/04/06 08:20:41 Range Scan of 1300 user documents took 36.786316ms
2016/04/06 08:20:41 PERFSTAT Range 0.0081 seconds
2016/04/06 08:20:41 Range Scan of 1300 user documents took 8.142751ms
2016/04/06 08:20:41 PERFSTAT Range 0.0033 seconds
2016/04/06 08:20:41 Range Scan of 1300 user documents took 3.302544ms
2016/04/06 08:20:41 PERFSTAT Range 0.0682 seconds
2016/04/06 08:20:41 Range Scan of 1400 user documents took 68.224506ms
2016/04/06 08:20:41 PERFSTAT Range 0.0005 seconds
2016/04/06 08:20:41 Range Scan of 1400 user documents took 509.656µs
2016/04/06 08:20:41 PERFSTAT Range 0.0720 seconds
2016/04/06 08:20:41 Range Scan of 1500 user documents took 72.027475ms
2016/04/06 08:20:41 PERFSTAT Range 0.0123 seconds
2016/04/06 08:20:41 Range Scan of 1500 user documents took 12.323186ms
2016/04/06 08:20:41 PERFSTAT Range 0.0253 seconds
2016/04/06 08:20:41 Range Scan of 1500 user documents took 25.367437ms
2016/04/06 08:20:41 PERFSTAT Range 0.0007 seconds
2016/04/06 08:20:41 Range Scan of 1500 user documents took 733.529µs
2016/04/06 08:20:41 PERFSTAT Range 0.0173 seconds
2016/04/06 08:20:41 Range Scan of 1500 user documents took 17.314553ms
2016/04/06 08:20:41 PERFSTAT Range 0.0005 seconds
2016/04/06 08:20:41 Range Scan of 1500 user documents took 501.865µs
2016/04/06 08:20:41 PERFSTAT Range 0.0006 seconds
2016/04/06 08:20:41 Range Scan of 1500 user documents took 603.422µs
2016/04/06 08:20:41 PERFSTAT Range 0.0909 seconds
2016/04/06 08:20:41 Range Scan of 1600 user documents took 90.956291ms
2016/04/06 08:20:41 PERFSTAT Range 0.0005 seconds
2016/04/06 08:20:41 Range Scan of 1600 user documents took 533.403µs
2016/04/06 08:20:41 PERFSTAT Range 0.0004 seconds
2016/04/06 08:20:41 Range Scan of 1600 user documents took 453.415µs
2016/04/06 08:20:41 PERFSTAT Range 0.0320 seconds
2016/04/06 08:20:41 Range Scan of 1600 user documents took 32.021298ms
2016/04/06 08:20:41 PERFSTAT Range 0.0096 seconds
2016/04/06 08:20:41 Range Scan of 1600 user documents took 9.673462ms
2016/04/06 08:20:41 PERFSTAT Range 0.0055 seconds
2016/04/06 08:20:41 Range Scan of 1600 user documents took 5.565805ms
2016/04/06 08:20:41 PERFSTAT Range 0.0206 seconds
2016/04/06 08:20:41 Range Scan of 1600 user documents took 21.388812ms
2016/04/06 08:20:41 PERFSTAT Range 0.0004 seconds
2016/04/06 08:20:41 Range Scan of 1600 user documents took 455.068µs
2016/04/06 08:20:41 PERFSTAT Range 0.0702 seconds
2016/04/06 08:20:41 Range Scan of 1600 user documents took 70.223379ms
2016/04/06 08:20:41 PERFSTAT Range 0.0246 seconds
2016/04/06 08:20:41 Range Scan of 1700 user documents took 24.621437ms
2016/04/06 08:20:41 PERFSTAT Range 0.0004 seconds
2016/04/06 08:20:41 Range Scan of 1700 user documents took 428.188µs
2016/04/06 08:20:41 PERFSTAT Range 0.0100 seconds
2016/04/06 08:20:41 Range Scan of 1700 user documents took 10.010614ms
2016/04/06 08:20:41 PERFSTAT Range 0.0005 seconds
2016/04/06 08:20:41 Range Scan of 1700 user documents took 484.566µs
2016/04/06 08:20:41 PERFSTAT Range 0.0004 seconds
2016/04/06 08:20:41 Range Scan of 1700 user documents took 443.823µs
2016/04/06 08:20:41 PERFSTAT Range 0.0666 seconds
2016/04/06 08:20:41 Range Scan of 1700 user documents took 66.76033ms
2016/04/06 08:20:41 PERFSTAT Range 0.0006 seconds
2016/04/06 08:20:41 Range Scan of 1700 user documents took 673.225µs
2016/04/06 08:20:41 PERFSTAT Range 0.0118 seconds
2016/04/06 08:20:41 Range Scan of 1700 user documents took 11.851667ms
2016/04/06 08:20:41 PERFSTAT Range 0.0006 seconds
2016/04/06 08:20:41 Range Scan of 1700 user documents took 624.479µs
2016/04/06 08:20:41 PERFSTAT Range 0.0013 seconds
2016/04/06 08:20:41 Range Scan of 1700 user documents took 1.346316ms
2016/04/06 08:20:41 PERFSTAT Range 0.0005 seconds
2016/04/06 08:20:41 Range Scan of 1700 user documents took 580.463µs
2016/04/06 08:20:42 PERFSTAT Range 0.1784 seconds
2016/04/06 08:20:42 Range Scan of 1900 user documents took 178.459805ms
2016/04/06 08:20:42 PERFSTAT Range 0.0609 seconds
2016/04/06 08:20:42 Range Scan of 1900 user documents took 60.920408ms
2016/04/06 08:20:42 PERFSTAT Range 0.0762 seconds
2016/04/06 08:20:42 Range Scan of 2000 user documents took 76.333727ms
2016/04/06 08:20:42 PERFSTAT Range 0.0495 seconds
2016/04/06 08:20:42 Range Scan of 2000 user documents took 49.565442ms
2016/04/06 08:20:42 PERFSTAT Range 0.0006 seconds
2016/04/06 08:20:42 Range Scan of 2000 user documents took 618.699µs
2016/04/06 08:20:42 PERFSTAT Range 0.0005 seconds
2016/04/06 08:20:42 Range Scan of 2000 user documents took 497.142µs
2016/04/06 08:20:42 PERFSTAT Range 0.0003 seconds
2016/04/06 08:20:42 Range Scan of 2000 user documents took 365.521µs
2016/04/06 08:20:42 PERFSTAT Range 0.0953 seconds
2016/04/06 08:20:42 Range Scan of 2000 user documents took 95.303632ms
2016/04/06 08:20:42 PERFSTAT Range 0.0005 seconds
2016/04/06 08:20:42 Range Scan of 2000 user documents took 526.216µs
2016/04/06 08:20:42 PERFSTAT Range 0.0915 seconds
2016/04/06 08:20:42 Range Scan of 2100 user documents took 91.54961ms
2016/04/06 08:20:42 PERFSTAT Range 0.0022 seconds
2016/04/06 08:20:42 Range Scan of 2100 user documents took 2.319018ms
2016/04/06 08:20:42 PERFSTAT Range 0.0004 seconds
2016/04/06 08:20:42 Range Scan of 2100 user documents took 386.317µs
2016/04/06 08:20:42 PERFSTAT Range 0.0003 seconds
2016/04/06 08:20:42 Range Scan of 2100 user documents took 368.276µs
2016/04/06 08:20:42 PERFSTAT Range 0.0014 seconds
2016/04/06 08:20:42 Range Scan of 2100 user documents took 1.41004ms
2016/04/06 08:20:42 PERFSTAT Range 0.0005 seconds
2016/04/06 08:20:42 Range Scan of 2100 user documents took 545.269µs
2016/04/06 08:20:42 PERFSTAT Range 0.0972 seconds
2016/04/06 08:20:42 Range Scan of 2100 user documents took 97.284164ms
2016/04/06 08:20:42 PERFSTAT Range 0.0005 seconds
2016/04/06 08:20:42 Range Scan of 2100 user documents took 577.907µs
2016/04/06 08:20:42 PERFSTAT Range 0.0004 seconds
2016/04/06 08:20:42 Range Scan of 2100 user documents took 409.009µs
2016/04/06 08:20:42 PERFSTAT Range 0.1184 seconds
2016/04/06 08:20:42 Range Scan of 2200 user documents took 118.458094ms
2016/04/06 08:20:42 PERFSTAT Range 0.0416 seconds
2016/04/06 08:20:42 Range Scan of 2300 user documents took 41.615643ms
2016/04/06 08:20:42 PERFSTAT Range 0.0012 seconds
2016/04/06 08:20:42 Range Scan of 2300 user documents took 1.215165ms
2016/04/06 08:20:42 PERFSTAT Range 0.0010 seconds
2016/04/06 08:20:42 Range Scan of 2300 user documents took 1.060062ms
2016/04/06 08:20:42 PERFSTAT Range 0.0004 seconds
2016/04/06 08:20:42 Range Scan of 2300 user documents took 417.212µs
2016/04/06 08:20:42 PERFSTAT Range 0.1329 seconds
2016/04/06 08:20:42 Range Scan of 2400 user documents took 132.911295ms
2016/04/06 08:20:42 PERFSTAT Range 0.0005 seconds
2016/04/06 08:20:42 Range Scan of 2400 user documents took 521.812µs
2016/04/06 08:20:43 PERFSTAT Range 0.1293 seconds
2016/04/06 08:20:43 Range Scan of 2400 user documents took 129.321206ms
2016/04/06 08:20:43 PERFSTAT Range 0.0116 seconds
2016/04/06 08:20:43 Range Scan of 2400 user documents took 11.671691ms
2016/04/06 08:20:43 PERFSTAT Range 0.0007 seconds
2016/04/06 08:20:43 Range Scan of 2400 user documents took 708.713µs
2016/04/06 08:20:43 PERFSTAT Range 0.0004 seconds
2016/04/06 08:20:43 Range Scan of 2400 user documents took 461.719µs
2016/04/06 08:20:43 PERFSTAT Range 0.0338 seconds
2016/04/06 08:20:43 Range Scan of 2400 user documents took 33.870803ms
2016/04/06 08:20:43 PERFSTAT Range 0.0690 seconds
2016/04/06 08:20:43 Range Scan of 2500 user documents took 69.026309ms
2016/04/06 08:20:43 PERFSTAT Range 0.0022 seconds
2016/04/06 08:20:43 Range Scan of 2500 user documents took 2.264945ms
2016/04/06 08:20:43 PERFSTAT Range 0.0040 seconds
2016/04/06 08:20:43 Range Scan of 2500 user documents took 4.046434ms
2016/04/06 08:20:43 PERFSTAT Range 0.0031 seconds
2016/04/06 08:20:43 Range Scan of 2500 user documents took 3.171579ms
2016/04/06 08:20:43 PERFSTAT Range 0.0007 seconds
2016/04/06 08:20:43 Range Scan of 2500 user documents took 723.635µs
2016/04/06 08:20:43 PERFSTAT Range 0.0023 seconds
2016/04/06 08:20:43 Range Scan of 2500 user documents took 2.35326ms
2016/04/06 08:20:43 PERFSTAT Range 0.0897 seconds
2016/04/06 08:20:43 Range Scan of 2500 user documents took 89.712394ms
2016/04/06 08:20:43 PERFSTAT Range 0.0018 seconds
2016/04/06 08:20:43 Range Scan of 2500 user documents took 1.817315ms
2016/04/06 08:20:43 PERFSTAT Range 0.0066 seconds
2016/04/06 08:20:43 Range Scan of 2500 user documents took 6.589107ms
2016/04/06 08:20:43 PERFSTAT Range 0.0700 seconds
2016/04/06 08:20:43 Range Scan of 2600 user documents took 70.086608ms
2016/04/06 08:20:43 PERFSTAT Range 0.0892 seconds
2016/04/06 08:20:43 Range Scan of 2700 user documents took 89.220926ms
2016/04/06 08:20:43 PERFSTAT Range 0.0005 seconds
2016/04/06 08:20:43 Range Scan of 2700 user documents took 500.669µs
2016/04/06 08:20:43 PERFSTAT Range 0.0007 seconds
2016/04/06 08:20:43 Range Scan of 2700 user documents took 682.339µs
2016/04/06 08:20:43 PERFSTAT Range 0.0496 seconds
2016/04/06 08:20:43 Range Scan of 2700 user documents took 49.683842ms
2016/04/06 08:20:43 PERFSTAT Range 0.0306 seconds
2016/04/06 08:20:43 Range Scan of 2700 user documents took 30.896111ms
2016/04/06 08:20:43 PERFSTAT Range 0.0012 seconds
2016/04/06 08:20:43 Range Scan of 2700 user documents took 1.209364ms
2016/04/06 08:20:43 PERFSTAT Range 0.0041 seconds
2016/04/06 08:20:43 Range Scan of 2700 user documents took 4.158199ms
2016/04/06 08:20:43 PERFSTAT Range 0.0127 seconds
2016/04/06 08:20:43 Range Scan of 2700 user documents took 12.716615ms
2016/04/06 08:20:43 PERFSTAT Range 0.0681 seconds
2016/04/06 08:20:43 Range Scan of 2800 user documents took 68.158227ms
2016/04/06 08:20:43 PERFSTAT Range 0.0282 seconds
2016/04/06 08:20:43 Range Scan of 2800 user documents took 28.267658ms
2016/04/06 08:20:43 PERFSTAT Range 0.0005 seconds
2016/04/06 08:20:43 Range Scan of 2800 user documents took 585.333µs
2016/04/06 08:20:43 PERFSTAT Range 0.0183 seconds
2016/04/06 08:20:43 Range Scan of 2800 user documents took 18.318313ms
2016/04/06 08:20:43 PERFSTAT Range 0.0036 seconds
2016/04/06 08:20:43 Range Scan of 2800 user documents took 3.905865ms
2016/04/06 08:20:43 PERFSTAT Range 0.0046 seconds
2016/04/06 08:20:43 Range Scan of 2800 user documents took 4.647918ms
2016/04/06 08:20:43 PERFSTAT Range 0.0813 seconds
2016/04/06 08:20:43 Range Scan of 2900 user documents took 81.299562ms
2016/04/06 08:20:43 PERFSTAT Range 0.0190 seconds
2016/04/06 08:20:43 Range Scan of 2900 user documents took 19.036271ms
2016/04/06 08:20:43 PERFSTAT Range 0.0090 seconds
2016/04/06 08:20:43 Range Scan of 2900 user documents took 9.056696ms
2016/04/06 08:20:43 PERFSTAT Range 0.0513 seconds
2016/04/06 08:20:43 Range Scan of 2900 user documents took 51.382155ms
2016/04/06 08:20:43 PERFSTAT Range 0.0064 seconds
2016/04/06 08:20:43 Range Scan of 2900 user documents took 6.435413ms
2016/04/06 08:20:43 PERFSTAT Range 0.1182 seconds
2016/04/06 08:20:43 Range Scan of 3000 user documents took 118.20568ms
2016/04/06 08:20:44 PERFSTAT Range 0.0742 seconds
2016/04/06 08:20:44 Range Scan of 3000 user documents took 74.268522ms
2016/04/06 08:20:44 PERFSTAT Range 0.0096 seconds
2016/04/06 08:20:44 Range Scan of 3000 user documents took 9.608611ms
2016/04/06 08:20:44 PERFSTAT Range 0.0004 seconds
2016/04/06 08:20:44 Range Scan of 3000 user documents took 407.576µs
2016/04/06 08:20:44 PERFSTAT Range 0.0515 seconds
2016/04/06 08:20:44 Range Scan of 3100 user documents took 51.506901ms
2016/04/06 08:20:44 PERFSTAT Range 0.0286 seconds
2016/04/06 08:20:44 Range Scan of 3100 user documents took 28.631548ms
2016/04/06 08:20:44 PERFSTAT Range 0.0006 seconds
2016/04/06 08:20:44 Range Scan of 3100 user documents took 607.555µs
2016/04/06 08:20:44 PERFSTAT Range 0.0004 seconds
2016/04/06 08:20:44 Range Scan of 3100 user documents took 401.695µs
2016/04/06 08:20:44 PERFSTAT Range 0.0202 seconds
2016/04/06 08:20:44 Range Scan of 3100 user documents took 20.363782ms
2016/04/06 08:20:44 PERFSTAT Range 0.0976 seconds
2016/04/06 08:20:44 Range Scan of 3200 user documents took 97.715977ms
2016/04/06 08:20:44 PERFSTAT Range 0.0046 seconds
2016/04/06 08:20:44 Range Scan of 3200 user documents took 4.662699ms
2016/04/06 08:20:44 PERFSTAT Range 0.0004 seconds
2016/04/06 08:20:44 Range Scan of 3200 user documents took 442.871µs
2016/04/06 08:20:44 PERFSTAT Range 0.0004 seconds
2016/04/06 08:20:44 Range Scan of 3200 user documents took 455.235µs
2016/04/06 08:20:44 PERFSTAT Range 0.0005 seconds
2016/04/06 08:20:44 Range Scan of 3200 user documents took 482.794µs
2016/04/06 08:20:44 PERFSTAT Range 0.0004 seconds
2016/04/06 08:20:44 Range Scan of 3200 user documents took 458.89µs
2016/04/06 08:20:44 PERFSTAT Range 0.0724 seconds
2016/04/06 08:20:44 Range Scan of 3200 user documents took 72.446281ms
2016/04/06 08:20:44 PERFSTAT Range 0.0005 seconds
2016/04/06 08:20:44 Range Scan of 3200 user documents took 497.824µs
2016/04/06 08:20:44 PERFSTAT Range 0.1166 seconds
2016/04/06 08:20:44 Range Scan of 3300 user documents took 116.683874ms
2016/04/06 08:20:44 PERFSTAT Range 0.0036 seconds
2016/04/06 08:20:44 Range Scan of 3300 user documents took 3.628831ms
2016/04/06 08:20:44 PERFSTAT Range 0.0005 seconds
2016/04/06 08:20:44 Range Scan of 3300 user documents took 513.556µs
2016/04/06 08:20:44 PERFSTAT Range 0.0005 seconds
2016/04/06 08:20:44 Range Scan of 3300 user documents took 3.334391ms
2016/04/06 08:20:44 PERFSTAT Range 0.0005 seconds
2016/04/06 08:20:44 Range Scan of 3300 user documents took 485.931µs
2016/04/06 08:20:44 PERFSTAT Range 0.0630 seconds
2016/04/06 08:20:44 Range Scan of 3300 user documents took 63.07889ms
2016/04/06 08:20:44 PERFSTAT Range 0.0549 seconds
2016/04/06 08:20:44 Range Scan of 3300 user documents took 54.940442ms
2016/04/06 08:20:44 PERFSTAT Range 0.0006 seconds
2016/04/06 08:20:44 Range Scan of 3300 user documents took 735.976µs
2016/04/06 08:20:44 PERFSTAT Range 0.0007 seconds
2016/04/06 08:20:44 Range Scan of 3300 user documents took 732.915µs
2016/04/06 08:20:44 PERFSTAT Range 0.1550 seconds
2016/04/06 08:20:44 Range Scan of 3500 user documents took 155.071833ms
2016/04/06 08:20:44 PERFSTAT Range 0.0005 seconds
2016/04/06 08:20:44 Range Scan of 3500 user documents took 524.936µs
2016/04/06 08:20:44 PERFSTAT Range 0.2039 seconds
2016/04/06 08:20:44 Range Scan of 3600 user documents took 203.943325ms
2016/04/06 08:20:44 PERFSTAT Range 0.0005 seconds
2016/04/06 08:20:44 Range Scan of 3600 user documents took 537.353µs
2016/04/06 08:20:44 PERFSTAT Range 0.0004 seconds
2016/04/06 08:20:44 Range Scan of 3600 user documents took 450.583µs
2016/04/06 08:20:44 PERFSTAT Range 0.0607 seconds
2016/04/06 08:20:44 Range Scan of 3700 user documents took 60.748074ms
2016/04/06 08:20:45 PERFSTAT Range 0.1290 seconds
2016/04/06 08:20:45 Range Scan of 3800 user documents took 129.029496ms
2016/04/06 08:20:45 PERFSTAT Range 0.0078 seconds
2016/04/06 08:20:45 Range Scan of 3800 user documents took 7.861829ms
2016/04/06 08:20:45 PERFSTAT Range 0.0004 seconds
2016/04/06 08:20:45 Range Scan of 3800 user documents took 454.014µs
2016/04/06 08:20:45 PERFSTAT Range 0.3000 seconds
2016/04/06 08:20:45 Range Scan of 4000 user documents took 299.99076ms
2016/04/06 08:20:45 PERFSTAT Range 0.2479 seconds
2016/04/06 08:20:45 Range Scan of 4100 user documents took 247.957402ms
2016/04/06 08:20:45 PERFSTAT Range 0.0028 seconds
2016/04/06 08:20:45 Range Scan of 4100 user documents took 2.807893ms
2016/04/06 08:20:45 PERFSTAT Range 0.0004 seconds
2016/04/06 08:20:45 Range Scan of 4100 user documents took 441.41µs
2016/04/06 08:20:45 PERFSTAT Range 0.0005 seconds
2016/04/06 08:20:45 Range Scan of 4100 user documents took 539.165µs
2016/04/06 08:20:45 PERFSTAT Range 0.0004 seconds
2016/04/06 08:20:45 Range Scan of 4100 user documents took 399.443µs
2016/04/06 08:20:45 PERFSTAT Range 0.1046 seconds
2016/04/06 08:20:45 Range Scan of 4300 user documents took 104.591736ms
2016/04/06 08:20:45 PERFSTAT Range 0.0005 seconds
2016/04/06 08:20:45 Range Scan of 4300 user documents took 527.401µs
2016/04/06 08:20:45 PERFSTAT Range 0.1193 seconds
2016/04/06 08:20:45 Range Scan of 4400 user documents took 119.37313ms
2016/04/06 08:20:45 PERFSTAT Range 0.0077 seconds
2016/04/06 08:20:45 Range Scan of 4400 user documents took 7.731654ms
2016/04/06 08:20:45 PERFSTAT Range 0.0005 seconds
2016/04/06 08:20:45 Range Scan of 4400 user documents took 499.09µs
2016/04/06 08:20:45 PERFSTAT Range 0.0031 seconds
2016/04/06 08:20:45 Range Scan of 4400 user documents took 3.156246ms
2016/04/06 08:20:45 PERFSTAT Range 0.0004 seconds
2016/04/06 08:20:45 Range Scan of 4400 user documents took 461.579µs
2016/04/06 08:20:45 PERFSTAT Range 0.0688 seconds
2016/04/06 08:20:45 Range Scan of 4400 user documents took 68.837308ms
2016/04/06 08:20:46 PERFSTAT Range 0.0773 seconds
2016/04/06 08:20:46 Range Scan of 4400 user documents took 77.330157ms
2016/04/06 08:20:46 PERFSTAT Range 0.0018 seconds
2016/04/06 08:20:46 Range Scan of 4400 user documents took 1.89461ms
2016/04/06 08:20:46 PERFSTAT Range 0.1904 seconds
2016/04/06 08:20:46 Range Scan of 4600 user documents took 190.431084ms
2016/04/06 08:20:46 PERFSTAT Range 0.0007 seconds
2016/04/06 08:20:46 Range Scan of 4600 user documents took 763.592µs
2016/04/06 08:20:46 PERFSTAT Range 0.0006 seconds
2016/04/06 08:20:46 Range Scan of 4600 user documents took 592.136µs
2016/04/06 08:20:46 PERFSTAT Range 0.0114 seconds
2016/04/06 08:20:46 Range Scan of 4600 user documents took 11.478717ms
2016/04/06 08:20:46 PERFSTAT Range 0.0004 seconds
2016/04/06 08:20:46 Range Scan of 4600 user documents took 472.547µs
2016/04/06 08:20:46 PERFSTAT Range 0.0016 seconds
2016/04/06 08:20:46 Range Scan of 4600 user documents took 1.621188ms
2016/04/06 08:20:46 PERFSTAT Range 0.0005 seconds
2016/04/06 08:20:46 Range Scan of 4600 user documents took 511.999µs
2016/04/06 08:20:46 PERFSTAT Range 0.0005 seconds
2016/04/06 08:20:46 Range Scan of 4600 user documents took 513.006µs
2016/04/06 08:20:46 PERFSTAT Range 0.0005 seconds
2016/04/06 08:20:46 Range Scan of 4600 user documents took 501.2µs
2016/04/06 08:20:46 PERFSTAT Range 0.0007 seconds
2016/04/06 08:20:46 Range Scan of 4600 user documents took 712.752µs
2016/04/06 08:20:46 PERFSTAT Range 0.1663 seconds
2016/04/06 08:20:46 Range Scan of 4700 user documents took 166.371748ms
2016/04/06 08:20:46 PERFSTAT Range 0.0663 seconds
2016/04/06 08:20:46 Range Scan of 4700 user documents took 66.438964ms
2016/04/06 08:20:46 PERFSTAT Range 0.0017 seconds
2016/04/06 08:20:46 Range Scan of 4700 user documents took 1.751297ms
2016/04/06 08:20:46 PERFSTAT Range 0.0006 seconds
2016/04/06 08:20:46 Range Scan of 4700 user documents took 632.131µs
2016/04/06 08:20:46 PERFSTAT Range 0.0005 seconds
2016/04/06 08:20:46 Range Scan of 4700 user documents took 776.281µs
2016/04/06 08:20:46 PERFSTAT Range 0.0015 seconds
2016/04/06 08:20:46 Range Scan of 4700 user documents took 1.677021ms
2016/04/06 08:20:46 PERFSTAT Range 0.0011 seconds
2016/04/06 08:20:46 Range Scan of 4700 user documents took 1.299027ms
2016/04/06 08:20:46 PERFSTAT Range 0.0021 seconds
2016/04/06 08:20:46 Range Scan of 4700 user documents took 2.132575ms
2016/04/06 08:20:46 PERFSTAT Range 0.0003 seconds
2016/04/06 08:20:46 Range Scan of 4700 user documents took 373.606µs
2016/04/06 08:20:46 PERFSTAT Range 0.1359 seconds
2016/04/06 08:20:46 Range Scan of 4800 user documents took 135.917388ms
2016/04/06 08:20:46 PERFSTAT Range 0.0035 seconds
2016/04/06 08:20:46 Range Scan of 4800 user documents took 3.579518ms
2016/04/06 08:20:46 PERFSTAT Range 0.2139 seconds
2016/04/06 08:20:46 Range Scan of 5000 user documents took 213.959217ms
2016/04/06 08:20:47 PERFSTAT Range 0.2396 seconds
2016/04/06 08:20:47 Range Scan of 5200 user documents took 239.636126ms
2016/04/06 08:20:47 PERFSTAT Range 0.0006 seconds
2016/04/06 08:20:47 Range Scan of 5200 user documents took 609.086µs
2016/04/06 08:20:47 PERFSTAT Range 0.0004 seconds
2016/04/06 08:20:47 Range Scan of 5200 user documents took 439.899µs
2016/04/06 08:20:47 PERFSTAT Range 0.0004 seconds
2016/04/06 08:20:47 Range Scan of 5200 user documents took 414.101µs
2016/04/06 08:20:47 PERFSTAT Range 0.1812 seconds
2016/04/06 08:20:47 Range Scan of 5300 user documents took 181.18982ms
2016/04/06 08:20:47 PERFSTAT Range 0.0028 seconds
2016/04/06 08:20:47 Range Scan of 5300 user documents took 2.856448ms
2016/04/06 08:20:47 PERFSTAT Range 0.1885 seconds
2016/04/06 08:20:47 Range Scan of 5400 user documents took 188.548102ms
2016/04/06 08:20:47 PERFSTAT Range 0.1436 seconds
2016/04/06 08:20:47 Range Scan of 5500 user documents took 143.602695ms
2016/04/06 08:20:48 PERFSTAT Range 0.5787 seconds
2016/04/06 08:20:48 Range Scan of 6000 user documents took 578.738073ms
2016/04/06 08:20:48 PERFSTAT Range 0.0006 seconds
2016/04/06 08:20:48 Range Scan of 6000 user documents took 614.688µs
2016/04/06 08:20:48 PERFSTAT Range 0.0005 seconds
2016/04/06 08:20:48 Range Scan of 6000 user documents took 552.389µs
2016/04/06 08:20:48 PERFSTAT Range 0.0004 seconds
2016/04/06 08:20:48 Range Scan of 6000 user documents took 430.893µs
2016/04/06 08:20:48 PERFSTAT Range 0.0004 seconds
2016/04/06 08:20:48 Range Scan of 6000 user documents took 406.943µs
2016/04/06 08:20:48 PERFSTAT Range 0.1939 seconds
2016/04/06 08:20:48 Range Scan of 6100 user documents took 193.962053ms
2016/04/06 08:20:48 PERFSTAT Range 0.0100 seconds
2016/04/06 08:20:48 Range Scan of 6100 user documents took 10.024514ms
2016/04/06 08:20:48 PERFSTAT Range 0.0394 seconds
2016/04/06 08:20:48 Range Scan of 6100 user documents took 39.457709ms
2016/04/06 08:20:48 PERFSTAT Range 0.0431 seconds
2016/04/06 08:20:48 Range Scan of 6200 user documents took 43.208046ms
2016/04/06 08:20:48 PERFSTAT Range 0.0006 seconds
2016/04/06 08:20:48 Range Scan of 6200 user documents took 645.091µs
2016/04/06 08:20:48 PERFSTAT Range 0.1613 seconds
2016/04/06 08:20:48 Range Scan of 6300 user documents took 161.436263ms
2016/04/06 08:20:48 PERFSTAT Range 0.3383 seconds
2016/04/06 08:20:48 Range Scan of 6500 user documents took 338.437491ms
2016/04/06 08:20:48 PERFSTAT Range 0.0020 seconds
2016/04/06 08:20:48 Range Scan of 6500 user documents took 2.128898ms
2016/04/06 08:20:49 PERFSTAT Range 0.1519 seconds
2016/04/06 08:20:49 Range Scan of 6600 user documents took 151.949459ms
2016/04/06 08:20:49 PERFSTAT Range 0.0989 seconds
2016/04/06 08:20:49 Range Scan of 6600 user documents took 98.949223ms
2016/04/06 08:20:49 PERFSTAT Range 0.0571 seconds
2016/04/06 08:20:49 Range Scan of 6700 user documents took 57.184763ms
2016/04/06 08:20:49 PERFSTAT Range 0.0006 seconds
2016/04/06 08:20:49 Range Scan of 6700 user documents took 632.054µs
2016/04/06 08:20:49 PERFSTAT Range 0.0006 seconds
2016/04/06 08:20:49 Range Scan of 6700 user documents took 625.426µs
2016/04/06 08:20:49 PERFSTAT Range 0.2464 seconds
2016/04/06 08:20:49 Range Scan of 6800 user documents took 246.432508ms
2016/04/06 08:20:49 PERFSTAT Range 0.0991 seconds
2016/04/06 08:20:49 Range Scan of 6900 user documents took 99.128125ms
2016/04/06 08:20:49 PERFSTAT Range 0.1046 seconds
2016/04/06 08:20:49 Range Scan of 6900 user documents took 104.716072ms
2016/04/06 08:20:49 PERFSTAT Range 0.1352 seconds
2016/04/06 08:20:49 Range Scan of 7000 user documents took 135.23804ms
2016/04/06 08:20:49 PERFSTAT Range 0.0191 seconds
2016/04/06 08:20:49 Range Scan of 7100 user documents took 19.402078ms
2016/04/06 08:20:49 PERFSTAT Range 0.0167 seconds
2016/04/06 08:20:49 Range Scan of 7100 user documents took 16.687194ms
2016/04/06 08:20:49 PERFSTAT Range 0.0004 seconds
2016/04/06 08:20:49 Range Scan of 7100 user documents took 383.343µs
2016/04/06 08:20:50 PERFSTAT Range 0.0902 seconds
2016/04/06 08:20:50 Range Scan of 7100 user documents took 90.199974ms
2016/04/06 08:20:50 PERFSTAT Range 0.1584 seconds
2016/04/06 08:20:50 Range Scan of 7200 user documents took 158.43591ms
2016/04/06 08:20:50 PERFSTAT Range 0.2727 seconds
2016/04/06 08:20:50 Range Scan of 7400 user documents took 272.933141ms
2016/04/06 08:20:50 PERFSTAT Range 0.0778 seconds
2016/04/06 08:20:50 Range Scan of 7500 user documents took 77.861607ms
2016/04/06 08:20:50 PERFSTAT Range 0.0005 seconds
2016/04/06 08:20:50 Range Scan of 7500 user documents took 552.602µs
2016/04/06 08:20:50 PERFSTAT Range 0.1133 seconds
2016/04/06 08:20:50 Range Scan of 7500 user documents took 113.426776ms
2016/04/06 08:20:50 PERFSTAT Range 0.0005 seconds
2016/04/06 08:20:50 Range Scan of 7500 user documents took 607.319µs
2016/04/06 08:20:50 PERFSTAT Range 0.0004 seconds
2016/04/06 08:20:50 Range Scan of 7500 user documents took 461.838µs
2016/04/06 08:20:50 PERFSTAT Range 0.0004 seconds
2016/04/06 08:20:50 Range Scan of 7500 user documents took 434.659µs
2016/04/06 08:20:50 PERFSTAT Range 0.0004 seconds
2016/04/06 08:20:50 Range Scan of 7500 user documents took 432.747µs
2016/04/06 08:20:50 PERFSTAT Range 0.0004 seconds
2016/04/06 08:20:50 Range Scan of 7500 user documents took 453.546µs
2016/04/06 08:20:50 PERFSTAT Range 0.0004 seconds
2016/04/06 08:20:50 Range Scan of 7500 user documents took 456.976µs
2016/04/06 08:20:50 PERFSTAT Range 0.0004 seconds
2016/04/06 08:20:50 Range Scan of 7500 user documents took 452.138µs
2016/04/06 08:20:50 PERFSTAT Range 0.1973 seconds
2016/04/06 08:20:50 Range Scan of 7600 user documents took 197.383423ms
2016/04/06 08:20:50 PERFSTAT Range 0.0005 seconds
2016/04/06 08:20:50 Range Scan of 7600 user documents took 521.797µs
2016/04/06 08:20:50 PERFSTAT Range 0.0003 seconds
2016/04/06 08:20:50 Range Scan of 7600 user documents took 326.021µs
2016/04/06 08:20:50 PERFSTAT Range 0.0003 seconds
2016/04/06 08:20:50 Range Scan of 7600 user documents took 336.586µs
2016/04/06 08:20:51 PERFSTAT Range 0.1736 seconds
2016/04/06 08:20:51 Range Scan of 7800 user documents took 173.638269ms
2016/04/06 08:20:51 PERFSTAT Range 0.0013 seconds
2016/04/06 08:20:51 Range Scan of 7800 user documents took 1.372835ms
2016/04/06 08:20:51 PERFSTAT Range 0.2013 seconds
2016/04/06 08:20:51 Range Scan of 7900 user documents took 201.380985ms
2016/04/06 08:20:51 PERFSTAT Range 0.0030 seconds
2016/04/06 08:20:51 Range Scan of 7900 user documents took 3.028445ms
2016/04/06 08:20:51 PERFSTAT Range 0.0044 seconds
2016/04/06 08:20:51 Range Scan of 7900 user documents took 4.450327ms
2016/04/06 08:20:51 PERFSTAT Range 0.0643 seconds
2016/04/06 08:20:51 Range Scan of 8000 user documents took 64.300532ms
2016/04/06 08:20:51 PERFSTAT Range 0.2577 seconds
2016/04/06 08:20:51 Range Scan of 8100 user documents took 257.791991ms
2016/04/06 08:20:51 PERFSTAT Range 0.0031 seconds
2016/04/06 08:20:51 Range Scan of 8100 user documents took 3.103325ms
2016/04/06 08:20:51 PERFSTAT Range 0.0902 seconds
2016/04/06 08:20:51 Range Scan of 8200 user documents took 90.244699ms
2016/04/06 08:20:51 PERFSTAT Range 0.1532 seconds
2016/04/06 08:20:51 Range Scan of 8300 user documents took 153.24379ms
2016/04/06 08:20:52 PERFSTAT Range 0.3153 seconds
2016/04/06 08:20:52 Range Scan of 8500 user documents took 315.301884ms
2016/04/06 08:20:52 PERFSTAT Range 0.0624 seconds
2016/04/06 08:20:52 Range Scan of 8600 user documents took 62.427373ms
2016/04/06 08:20:52 PERFSTAT Range 0.0081 seconds
2016/04/06 08:20:52 Range Scan of 8600 user documents took 8.210242ms
2016/04/06 08:20:52 PERFSTAT Range 0.0010 seconds
2016/04/06 08:20:52 Range Scan of 8600 user documents took 1.04248ms
2016/04/06 08:20:52 PERFSTAT Range 0.0155 seconds
2016/04/06 08:20:52 Range Scan of 8600 user documents took 15.514888ms
2016/04/06 08:20:52 PERFSTAT Range 0.0005 seconds
2016/04/06 08:20:52 Range Scan of 8600 user documents took 499.238µs
2016/04/06 08:20:52 PERFSTAT Range 0.0100 seconds
2016/04/06 08:20:52 Range Scan of 8600 user documents took 10.055003ms
2016/04/06 08:20:52 PERFSTAT Range 0.1725 seconds
2016/04/06 08:20:52 Range Scan of 8700 user documents took 172.51309ms
2016/04/06 08:20:52 PERFSTAT Range 0.0006 seconds
2016/04/06 08:20:52 Range Scan of 8700 user documents took 607.033µs
2016/04/06 08:20:52 PERFSTAT Range 0.4552 seconds
2016/04/06 08:20:52 Range Scan of 9000 user documents took 455.226855ms
2016/04/06 08:20:52 PERFSTAT Range 0.0019 seconds
2016/04/06 08:20:52 Range Scan of 9000 user documents took 1.993648ms
2016/04/06 08:20:53 PERFSTAT Range 0.3138 seconds
2016/04/06 08:20:53 Range Scan of 9200 user documents took 313.862185ms
2016/04/06 08:20:53 PERFSTAT Range 0.0073 seconds
2016/04/06 08:20:53 Range Scan of 9200 user documents took 7.314667ms
2016/04/06 08:20:53 PERFSTAT Range 0.0011 seconds
2016/04/06 08:20:53 Range Scan of 9200 user documents took 1.14025ms
2016/04/06 08:20:53 PERFSTAT Range 0.0004 seconds
2016/04/06 08:20:53 Range Scan of 9200 user documents took 395.731µs
2016/04/06 08:20:53 PERFSTAT Range 0.0005 seconds
2016/04/06 08:20:53 Range Scan of 9200 user documents took 494.562µs
2016/04/06 08:20:53 PERFSTAT Range 0.1976 seconds
2016/04/06 08:20:53 Range Scan of 9300 user documents took 197.639768ms
2016/04/06 08:20:53 PERFSTAT Range 0.1126 seconds
2016/04/06 08:20:53 Range Scan of 9400 user documents took 112.614667ms
2016/04/06 08:20:53 PERFSTAT Range 0.0010 seconds
2016/04/06 08:20:53 Range Scan of 9400 user documents took 1.057696ms
2016/04/06 08:20:53 PERFSTAT Range 0.3563 seconds
2016/04/06 08:20:53 Range Scan of 9700 user documents took 356.363072ms
2016/04/06 08:20:54 PERFSTAT Range 0.2683 seconds
2016/04/06 08:20:54 Range Scan of 9800 user documents took 268.478283ms
2016/04/06 08:20:54 PERFSTAT Range 0.1774 seconds
2016/04/06 08:20:54 Range Scan of 10000 user documents took 177.469155ms
2016/04/06 08:20:54 PERFSTAT Range 0.0006 seconds
2016/04/06 08:20:54 Range Scan of 10000 user documents took 671.236µs
2016/04/06 08:20:54 PERFSTAT Range 0.4510 seconds
2016/04/06 08:20:54 Range Scan of 10300 user documents took 451.126008ms
2016/04/06 08:20:54 PERFSTAT Range 0.0004 seconds
2016/04/06 08:20:54 Range Scan of 10300 user documents took 473.89µs
2016/04/06 08:20:54 PERFSTAT Range 0.0004 seconds
2016/04/06 08:20:54 Range Scan of 10300 user documents took 478.606µs
2016/04/06 08:20:54 PERFSTAT Range 0.0004 seconds
2016/04/06 08:20:54 Range Scan of 10300 user documents took 436.754µs
2016/04/06 08:20:54 PERFSTAT Range 0.0004 seconds
2016/04/06 08:20:54 Range Scan of 10300 user documents took 403.927µs
2016/04/06 08:20:54 PERFSTAT Range 0.0005 seconds
2016/04/06 08:20:54 Range Scan of 10300 user documents took 483.544µs
2016/04/06 08:20:55 PERFSTAT Range 0.4906 seconds
2016/04/06 08:20:55 Range Scan of 10600 user documents took 490.607753ms
2016/04/06 08:20:55 PERFSTAT Range 0.0914 seconds
2016/04/06 08:20:55 Range Scan of 10700 user documents took 91.437429ms
2016/04/06 08:20:55 PERFSTAT Range 0.1366 seconds
2016/04/06 08:20:55 Range Scan of 10800 user documents took 136.705489ms
2016/04/06 08:20:55 PERFSTAT Range 0.0404 seconds
2016/04/06 08:20:55 Range Scan of 10800 user documents took 40.47577ms
2016/04/06 08:20:55 PERFSTAT Range 0.2112 seconds
2016/04/06 08:20:55 Range Scan of 10900 user documents took 211.236877ms
2016/04/06 08:20:55 PERFSTAT Range 0.1948 seconds
2016/04/06 08:20:55 Range Scan of 11000 user documents took 194.85907ms
2016/04/06 08:20:55 PERFSTAT Range 0.0005 seconds
2016/04/06 08:20:55 Range Scan of 11000 user documents took 589.82µs
2016/04/06 08:20:56 PERFSTAT Range 0.3574 seconds
2016/04/06 08:20:56 Range Scan of 11300 user documents took 357.433898ms
2016/04/06 08:20:56 PERFSTAT Range 0.0005 seconds
2016/04/06 08:20:56 Range Scan of 11300 user documents took 550.314µs
2016/04/06 08:20:56 PERFSTAT Range 0.0005 seconds
2016/04/06 08:20:56 Range Scan of 11300 user documents took 505.866µs
2016/04/06 08:20:56 PERFSTAT Range 0.0008 seconds
2016/04/06 08:20:56 Range Scan of 11300 user documents took 815.47µs
2016/04/06 08:20:56 PERFSTAT Range 0.0757 seconds
2016/04/06 08:20:56 Range Scan of 11400 user documents took 75.739715ms
2016/04/06 08:20:56 PERFSTAT Range 0.0006 seconds
2016/04/06 08:20:56 Range Scan of 11400 user documents took 600.289µs
2016/04/06 08:20:56 PERFSTAT Range 0.0004 seconds
2016/04/06 08:20:56 Range Scan of 11400 user documents took 466.98µs
2016/04/06 08:20:56 PERFSTAT Range 0.3427 seconds
2016/04/06 08:20:56 Range Scan of 11600 user documents took 342.756561ms
2016/04/06 08:20:56 PERFSTAT Range 0.0127 seconds
2016/04/06 08:20:56 Range Scan of 11600 user documents took 12.722377ms
2016/04/06 08:20:56 PERFSTAT Range 0.1112 seconds
2016/04/06 08:20:56 Range Scan of 11700 user documents took 111.255137ms
2016/04/06 08:20:56 PERFSTAT Range 0.0026 seconds
2016/04/06 08:20:56 Range Scan of 11700 user documents took 2.593795ms
2016/04/06 08:20:56 PERFSTAT Range 0.0072 seconds
2016/04/06 08:20:56 Range Scan of 11700 user documents took 7.260509ms
2016/04/06 08:20:57 PERFSTAT Range 0.2192 seconds
2016/04/06 08:20:57 Range Scan of 11800 user documents took 219.215451ms
2016/04/06 08:20:57 PERFSTAT Range 0.5767 seconds
2016/04/06 08:20:57 Range Scan of 12200 user documents took 576.774672ms
2016/04/06 08:20:57 PERFSTAT Range 0.0047 seconds
2016/04/06 08:20:57 Range Scan of 12200 user documents took 4.751969ms
2016/04/06 08:20:57 PERFSTAT Range 0.1944 seconds
2016/04/06 08:20:57 Range Scan of 12400 user documents took 194.402081ms
2016/04/06 08:20:57 PERFSTAT Range 0.0016 seconds
2016/04/06 08:20:57 Range Scan of 12400 user documents took 1.733473ms
2016/04/06 08:20:57 PERFSTAT Range 0.0068 seconds
2016/04/06 08:20:57 Range Scan of 12400 user documents took 6.831223ms
2016/04/06 08:20:58 PERFSTAT Range 0.2593 seconds
2016/04/06 08:20:58 Range Scan of 12500 user documents took 259.3835ms
2016/04/06 08:20:58 PERFSTAT Range 0.0009 seconds
2016/04/06 08:20:58 Range Scan of 12500 user documents took 953.434µs
2016/04/06 08:20:58 PERFSTAT Range 0.0005 seconds
2016/04/06 08:20:58 Range Scan of 12500 user documents took 541.934µs
2016/04/06 08:20:58 PERFSTAT Range 0.0005 seconds
2016/04/06 08:20:58 Range Scan of 12500 user documents took 529.242µs
2016/04/06 08:20:58 PERFSTAT Range 0.2504 seconds
2016/04/06 08:20:58 Range Scan of 12700 user documents took 250.458201ms
2016/04/06 08:20:58 PERFSTAT Range 0.4236 seconds
2016/04/06 08:20:58 Range Scan of 13000 user documents took 423.637339ms
2016/04/06 08:20:58 PERFSTAT Range 0.0005 seconds
2016/04/06 08:20:58 Range Scan of 13000 user documents took 518.8µs
2016/04/06 08:20:58 PERFSTAT Range 0.0011 seconds
2016/04/06 08:20:58 Range Scan of 13000 user documents took 1.123718ms
2016/04/06 08:20:58 PERFSTAT Range 0.0004 seconds
2016/04/06 08:20:58 Range Scan of 13000 user documents took 381.454µs
2016/04/06 08:20:58 PERFSTAT Range 0.0011 seconds
2016/04/06 08:20:58 Range Scan of 13000 user documents took 1.091424ms
2016/04/06 08:20:58 PERFSTAT Range 0.1044 seconds
2016/04/06 08:20:58 Range Scan of 13100 user documents took 104.448952ms
2016/04/06 08:20:58 PERFSTAT Range 0.0006 seconds
2016/04/06 08:20:58 Range Scan of 13100 user documents took 659.311µs
2016/04/06 08:20:58 PERFSTAT Range 0.0008 seconds
2016/04/06 08:20:58 Range Scan of 13100 user documents took 870.164µs
2016/04/06 08:20:58 PERFSTAT Range 0.0040 seconds
2016/04/06 08:20:58 Range Scan of 13100 user documents took 4.036992ms
2016/04/06 08:20:58 PERFSTAT Range 0.0004 seconds
2016/04/06 08:20:58 Range Scan of 13100 user documents took 385.272µs
2016/04/06 08:20:58 PERFSTAT Range 0.1062 seconds
2016/04/06 08:20:58 Range Scan of 13200 user documents took 111.3281ms
2016/04/06 08:20:59 PERFSTAT Range 0.2691 seconds
2016/04/06 08:20:59 Range Scan of 13300 user documents took 270.28627ms
2016/04/06 08:20:59 PERFSTAT Range 0.0005 seconds
2016/04/06 08:20:59 Range Scan of 13300 user documents took 577.986µs
2016/04/06 08:20:59 PERFSTAT Range 0.0004 seconds
2016/04/06 08:20:59 Range Scan of 13300 user documents took 464.862µs
2016/04/06 08:20:59 PERFSTAT Range 0.0006 seconds
2016/04/06 08:20:59 Range Scan of 13300 user documents took 700.986µs
2016/04/06 08:20:59 PERFSTAT Range 0.6202 seconds
2016/04/06 08:20:59 Range Scan of 13700 user documents took 620.273942ms
2016/04/06 08:21:00 PERFSTAT Range 0.2662 seconds
2016/04/06 08:21:00 Range Scan of 13900 user documents took 266.257399ms
2016/04/06 08:21:00 PERFSTAT Range 0.0603 seconds
2016/04/06 08:21:00 Range Scan of 13900 user documents took 60.438692ms
2016/04/06 08:21:00 PERFSTAT Range 0.4172 seconds
2016/04/06 08:21:00 Range Scan of 14200 user documents took 417.202013ms
2016/04/06 08:21:00 PERFSTAT Range 0.0078 seconds
2016/04/06 08:21:00 Range Scan of 14200 user documents took 7.788748ms
2016/04/06 08:21:00 PERFSTAT Range 0.0004 seconds
2016/04/06 08:21:00 Range Scan of 14200 user documents took 420.992µs
2016/04/06 08:21:00 PERFSTAT Range 0.0004 seconds
2016/04/06 08:21:00 Range Scan of 14200 user documents took 408.843µs
2016/04/06 08:21:00 PERFSTAT Range 0.2309 seconds
2016/04/06 08:21:00 Range Scan of 14300 user documents took 230.954969ms
2016/04/06 08:21:00 PERFSTAT Range 0.0005 seconds
2016/04/06 08:21:00 Range Scan of 14300 user documents took 543.344µs
2016/04/06 08:21:01 PERFSTAT Range 0.3290 seconds
2016/04/06 08:21:01 Range Scan of 14500 user documents took 329.014558ms
2016/04/06 08:21:01 PERFSTAT Range 0.2128 seconds
2016/04/06 08:21:01 Range Scan of 14700 user documents took 212.849235ms
2016/04/06 08:21:02 PERFSTAT Range 0.6562 seconds
2016/04/06 08:21:02 Range Scan of 15100 user documents took 656.20189ms
2016/04/06 08:21:02 PERFSTAT Range 0.0008 seconds
2016/04/06 08:21:02 Range Scan of 15100 user documents took 829.402µs
2016/04/06 08:21:02 PERFSTAT Range 0.0005 seconds
2016/04/06 08:21:02 Range Scan of 15100 user documents took 524.663µs
2016/04/06 08:21:02 PERFSTAT Range 0.0017 seconds
2016/04/06 08:21:02 Range Scan of 15100 user documents took 1.762469ms
2016/04/06 08:21:02 PERFSTAT Range 0.0011 seconds
2016/04/06 08:21:02 Range Scan of 15100 user documents took 1.173623ms
2016/04/06 08:21:02 PERFSTAT Range 0.8327 seconds
2016/04/06 08:21:02 Range Scan of 15600 user documents took 832.775002ms
2016/04/06 08:21:02 PERFSTAT Range 0.0006 seconds
2016/04/06 08:21:02 Range Scan of 15600 user documents took 605.197µs
2016/04/06 08:21:03 PERFSTAT Range 0.1744 seconds
2016/04/06 08:21:03 Range Scan of 15800 user documents took 174.491178ms
2016/04/06 08:21:03 PERFSTAT Range 0.2142 seconds
2016/04/06 08:21:03 Range Scan of 15900 user documents took 214.26542ms
2016/04/06 08:21:03 PERFSTAT Range 0.0005 seconds
2016/04/06 08:21:03 Range Scan of 15900 user documents took 513.353µs
2016/04/06 08:21:03 PERFSTAT Range 0.0004 seconds
2016/04/06 08:21:03 Range Scan of 15900 user documents took 420.078µs
2016/04/06 08:21:03 PERFSTAT Range 0.0562 seconds
2016/04/06 08:21:03 Range Scan of 15900 user documents took 56.282905ms
2016/04/06 08:21:03 PERFSTAT Range 0.0005 seconds
2016/04/06 08:21:03 Range Scan of 15900 user documents took 574.737µs
2016/04/06 08:21:03 PERFSTAT Range 0.3647 seconds
2016/04/06 08:21:03 Range Scan of 16100 user documents took 364.866696ms
2016/04/06 08:21:04 PERFSTAT Range 0.3243 seconds
2016/04/06 08:21:04 Range Scan of 16300 user documents took 324.319465ms
2016/04/06 08:21:04 PERFSTAT Range 0.0005 seconds
2016/04/06 08:21:04 Range Scan of 16300 user documents took 532.68µs
2016/04/06 08:21:04 PERFSTAT Range 0.2331 seconds
2016/04/06 08:21:04 Range Scan of 16500 user documents took 233.144356ms
2016/04/06 08:21:04 PERFSTAT Range 0.3832 seconds
2016/04/06 08:21:04 Range Scan of 16800 user documents took 384.078271ms
2016/04/06 08:21:04 PERFSTAT Range 0.1011 seconds
2016/04/06 08:21:04 Range Scan of 16900 user documents took 101.135743ms
2016/04/06 08:21:04 PERFSTAT Range 0.0005 seconds
2016/04/06 08:21:04 Range Scan of 16900 user documents took 557.724µs
2016/04/06 08:21:04 PERFSTAT Range 0.0428 seconds
2016/04/06 08:21:04 Range Scan of 16900 user documents took 42.881747ms
2016/04/06 08:21:04 PERFSTAT Range 0.0011 seconds
2016/04/06 08:21:04 Range Scan of 16900 user documents took 2.149774ms
2016/04/06 08:21:04 PERFSTAT Range 0.0418 seconds
2016/04/06 08:21:04 Range Scan of 16900 user documents took 41.889768ms
2016/04/06 08:21:05 PERFSTAT Range 0.2977 seconds
2016/04/06 08:21:05 Range Scan of 17100 user documents took 297.865976ms
2016/04/06 08:21:05 PERFSTAT Range 0.0007 seconds
2016/04/06 08:21:05 Range Scan of 17100 user documents took 769.144µs
2016/04/06 08:21:05 PERFSTAT Range 0.7750 seconds
2016/04/06 08:21:05 Range Scan of 17700 user documents took 775.055527ms
2016/04/06 08:21:05 PERFSTAT Range 0.0005 seconds
2016/04/06 08:21:05 Range Scan of 17700 user documents took 566.938µs
2016/04/06 08:21:06 PERFSTAT Range 0.1517 seconds
2016/04/06 08:21:06 Range Scan of 17800 user documents took 151.800784ms
2016/04/06 08:21:06 PERFSTAT Range 0.0008 seconds
2016/04/06 08:21:06 Range Scan of 17800 user documents took 890.12µs
2016/04/06 08:21:06 PERFSTAT Range 0.0160 seconds
2016/04/06 08:21:06 Range Scan of 17800 user documents took 16.126983ms
2016/04/06 08:21:06 PERFSTAT Range 0.0024 seconds
2016/04/06 08:21:06 Range Scan of 17800 user documents took 2.418706ms
2016/04/06 08:21:06 PERFSTAT Range 0.0783 seconds
2016/04/06 08:21:06 Range Scan of 17800 user documents took 78.347616ms
2016/04/06 08:21:06 PERFSTAT Range 0.6149 seconds
2016/04/06 08:21:06 Range Scan of 18300 user documents took 614.89645ms
2016/04/06 08:21:07 PERFSTAT Range 0.3531 seconds
2016/04/06 08:21:07 Range Scan of 18500 user documents took 353.151602ms
2016/04/06 08:21:07 PERFSTAT Range 0.0005 seconds
2016/04/06 08:21:07 Range Scan of 18500 user documents took 486.425µs
2016/04/06 08:21:07 PERFSTAT Range 0.5197 seconds
2016/04/06 08:21:07 Range Scan of 18800 user documents took 519.745651ms
2016/04/06 08:21:07 PERFSTAT Range 0.0004 seconds
2016/04/06 08:21:07 Range Scan of 18800 user documents took 478.881µs
2016/04/06 08:21:08 PERFSTAT Range 0.8034 seconds
2016/04/06 08:21:08 Range Scan of 19400 user documents took 803.442174ms
2016/04/06 08:21:08 PERFSTAT Range 0.0028 seconds
2016/04/06 08:21:08 Range Scan of 19400 user documents took 2.869663ms
2016/04/06 08:21:08 PERFSTAT Range 0.0004 seconds
2016/04/06 08:21:08 Range Scan of 19400 user documents took 427.091µs
2016/04/06 08:21:08 PERFSTAT Range 0.0004 seconds
2016/04/06 08:21:08 Range Scan of 19400 user documents took 683.85µs
2016/04/06 08:21:08 PERFSTAT Range 0.0009 seconds
2016/04/06 08:21:08 Range Scan of 19400 user documents took 910.14µs
2016/04/06 08:21:08 PERFSTAT Range 0.1992 seconds
2016/04/06 08:21:08 Range Scan of 19500 user documents took 199.27348ms
2016/04/06 08:21:08 PERFSTAT Range 0.0005 seconds
2016/04/06 08:21:08 Range Scan of 19500 user documents took 564.728µs
2016/04/06 08:21:08 PERFSTAT Range 0.0013 seconds
2016/04/06 08:21:08 Range Scan of 19500 user documents took 1.288087ms
2016/04/06 08:21:08 PERFSTAT Range 0.0004 seconds
2016/04/06 08:21:08 Range Scan of 19500 user documents took 425.41µs
2016/04/06 08:21:08 PERFSTAT Range 0.0008 seconds
2016/04/06 08:21:08 Range Scan of 19500 user documents took 858.136µs
2016/04/06 08:21:08 PERFSTAT Range 0.0006 seconds
2016/04/06 08:21:08 Range Scan of 19500 user documents took 631.185µs
2016/04/06 08:21:08 PERFSTAT Range 0.0005 seconds
2016/04/06 08:21:08 Range Scan of 19500 user documents took 526.209µs
2016/04/06 08:21:09 PERFSTAT Range 0.4909 seconds
2016/04/06 08:21:09 Range Scan of 19800 user documents took 490.913537ms
2016/04/06 08:21:09 PERFSTAT Range 0.0006 seconds
2016/04/06 08:21:09 Range Scan of 19800 user documents took 662.328µs
2016/04/06 08:21:09 PERFSTAT Range 0.0006 seconds
2016/04/06 08:21:09 Range Scan of 19800 user documents took 625.709µs
2016/04/06 08:21:09 PERFSTAT Range 0.4437 seconds
2016/04/06 08:21:09 Range Scan of 20200 user documents took 443.779394ms
2016/04/06 08:21:09 PERFSTAT Range 0.0005 seconds
2016/04/06 08:21:09 Range Scan of 20200 user documents took 558.418µs
2016/04/06 08:21:09 PERFSTAT Range 0.0005 seconds
2016/04/06 08:21:09 Range Scan of 20200 user documents took 529.189µs
2016/04/06 08:21:09 PERFSTAT Range 0.0005 seconds
2016/04/06 08:21:09 Range Scan of 20200 user documents took 506.127µs
2016/04/06 08:21:09 PERFSTAT Range 0.0005 seconds
2016/04/06 08:21:09 Range Scan of 20200 user documents took 533.572µs
2016/04/06 08:21:09 PERFSTAT Range 0.1389 seconds
2016/04/06 08:21:09 Range Scan of 20200 user documents took 138.899884ms
2016/04/06 08:21:09 PERFSTAT Range 0.0006 seconds
2016/04/06 08:21:09 Range Scan of 20200 user documents took 672.574µs
2016/04/06 08:21:09 PERFSTAT Range 0.0007 seconds
2016/04/06 08:21:09 Range Scan of 20200 user documents took 733.014µs
2016/04/06 08:21:09 PERFSTAT Range 0.0004 seconds
2016/04/06 08:21:09 Range Scan of 20200 user documents took 510.334µs
2016/04/06 08:21:10 PERFSTAT Range 1.0098 seconds
2016/04/06 08:21:10 Range Scan of 20900 user documents took 1.009872485s
2016/04/06 08:21:10 PERFSTAT Range 0.0005 seconds
2016/04/06 08:21:10 Range Scan of 20900 user documents took 480.355µs
2016/04/06 08:21:10 PERFSTAT Range 0.1272 seconds
2016/04/06 08:21:10 Range Scan of 21000 user documents took 127.203771ms
2016/04/06 08:21:10 PERFSTAT Range 0.0026 seconds
2016/04/06 08:21:10 Range Scan of 21000 user documents took 2.630638ms
2016/04/06 08:21:11 PERFSTAT Range 0.5594 seconds
2016/04/06 08:21:11 Range Scan of 21400 user documents took 563.051304ms
2016/04/06 08:21:11 PERFSTAT Range 0.0020 seconds
2016/04/06 08:21:11 Range Scan of 21400 user documents took 2.047869ms
2016/04/06 08:21:11 PERFSTAT Range 0.2908 seconds
2016/04/06 08:21:11 Range Scan of 21600 user documents took 291.654966ms
2016/04/06 08:21:11 PERFSTAT Range 0.0010 seconds
2016/04/06 08:21:11 Range Scan of 21600 user documents took 1.023868ms
2016/04/06 08:21:11 PERFSTAT Range 0.2191 seconds
2016/04/06 08:21:11 Range Scan of 21700 user documents took 219.177388ms
2016/04/06 08:21:12 PERFSTAT Range 0.2526 seconds
2016/04/06 08:21:12 Range Scan of 21900 user documents took 252.686978ms
2016/04/06 08:21:12 PERFSTAT Range 0.0005 seconds
2016/04/06 08:21:12 Range Scan of 21900 user documents took 490.902µs
2016/04/06 08:21:12 PERFSTAT Range 0.0006 seconds
2016/04/06 08:21:12 Range Scan of 21900 user documents took 640.628µs
2016/04/06 08:21:12 PERFSTAT Range 0.2145 seconds
2016/04/06 08:21:12 Range Scan of 22000 user documents took 214.533601ms
2016/04/06 08:21:12 PERFSTAT Range 0.1089 seconds
2016/04/06 08:21:12 Range Scan of 22100 user documents took 109.06179ms
2016/04/06 08:21:12 PERFSTAT Range 0.0006 seconds
2016/04/06 08:21:12 Range Scan of 22100 user documents took 688.207µs
2016/04/06 08:21:12 PERFSTAT Range 0.4570 seconds
2016/04/06 08:21:12 Range Scan of 22500 user documents took 457.07233ms
2016/04/06 08:21:13 PERFSTAT Range 0.1688 seconds
2016/04/06 08:21:13 Range Scan of 22600 user documents took 168.89153ms
2016/04/06 08:21:13 PERFSTAT Range 0.0019 seconds
2016/04/06 08:21:13 Range Scan of 22600 user documents took 1.896889ms
2016/04/06 08:21:13 PERFSTAT Range 0.0006 seconds
2016/04/06 08:21:13 Range Scan of 22600 user documents took 604.225µs
2016/04/06 08:21:14 PERFSTAT Range 1.0752 seconds
2016/04/06 08:21:14 Range Scan of 23300 user documents took 1.075258526s
2016/04/06 08:21:15 PERFSTAT Range 0.9081 seconds
2016/04/06 08:21:15 Range Scan of 24000 user documents took 908.17933ms
2016/04/06 08:21:15 PERFSTAT Range 0.7836 seconds
2016/04/06 08:21:15 Range Scan of 24500 user documents took 783.684404ms
2016/04/06 08:21:16 PERFSTAT Range 0.2313 seconds
2016/04/06 08:21:16 Range Scan of 24700 user documents took 231.393342ms
2016/04/06 08:21:16 PERFSTAT Range 0.0007 seconds
2016/04/06 08:21:16 Range Scan of 24700 user documents took 712.191µs
2016/04/06 08:21:16 PERFSTAT Range 0.3682 seconds
2016/04/06 08:21:16 Range Scan of 24900 user documents took 368.260989ms
2016/04/06 08:21:16 PERFSTAT Range 0.2907 seconds
2016/04/06 08:21:16 Range Scan of 25100 user documents took 290.783835ms
2016/04/06 08:21:17 PERFSTAT Range 0.2031 seconds
2016/04/06 08:21:17 Range Scan of 25300 user documents took 203.17118ms
2016/04/06 08:21:17 PERFSTAT Range 0.3237 seconds
2016/04/06 08:21:17 Range Scan of 25500 user documents took 323.775916ms
2016/04/06 08:21:17 PERFSTAT Range 0.3214 seconds
2016/04/06 08:21:17 Range Scan of 25700 user documents took 321.48969ms
2016/04/06 08:21:17 PERFSTAT Range 0.0785 seconds
2016/04/06 08:21:17 Range Scan of 25800 user documents took 78.583639ms
2016/04/06 08:21:17 PERFSTAT Range 0.0018 seconds
2016/04/06 08:21:17 Range Scan of 25800 user documents took 1.866597ms
2016/04/06 08:21:17 PERFSTAT Range 0.0014 seconds
2016/04/06 08:21:17 Range Scan of 25800 user documents took 1.409421ms
2016/04/06 08:21:17 PERFSTAT Range 0.0007 seconds
2016/04/06 08:21:17 Range Scan of 25800 user documents took 738.231µs
2016/04/06 08:21:18 PERFSTAT Range 0.5037 seconds
2016/04/06 08:21:18 Range Scan of 26100 user documents took 503.799687ms
2016/04/06 08:21:18 PERFSTAT Range 0.0008 seconds
2016/04/06 08:21:18 Range Scan of 26100 user documents took 797.686µs
2016/04/06 08:21:18 PERFSTAT Range 0.0004 seconds
2016/04/06 08:21:18 Range Scan of 26100 user documents took 541.316µs
2016/04/06 08:21:18 PERFSTAT Range 0.0021 seconds
2016/04/06 08:21:18 Range Scan of 26100 user documents took 2.180694ms
2016/04/06 08:21:18 PERFSTAT Range 0.0005 seconds
2016/04/06 08:21:18 Range Scan of 26100 user documents took 535.797µs
2016/04/06 08:21:18 PERFSTAT Range 0.0005 seconds
2016/04/06 08:21:18 Range Scan of 26100 user documents took 522.033µs
2016/04/06 08:21:18 PERFSTAT Range 0.0006 seconds
2016/04/06 08:21:18 Range Scan of 26100 user documents took 681.194µs
2016/04/06 08:21:18 PERFSTAT Range 0.0006 seconds
2016/04/06 08:21:18 Range Scan of 26100 user documents took 612.31µs
2016/04/06 08:21:18 PERFSTAT Range 0.6153 seconds
2016/04/06 08:21:18 Range Scan of 26500 user documents took 615.505123ms
2016/04/06 08:21:19 PERFSTAT Range 0.9263 seconds
2016/04/06 08:21:19 Range Scan of 27200 user documents took 926.377895ms
2016/04/06 08:21:19 PERFSTAT Range 0.0008 seconds
2016/04/06 08:21:19 Range Scan of 27200 user documents took 805.528µs
2016/04/06 08:21:20 PERFSTAT Range 0.3378 seconds
2016/04/06 08:21:20 Range Scan of 27400 user documents took 337.803367ms
2016/04/06 08:21:20 PERFSTAT Range 0.0005 seconds
2016/04/06 08:21:20 Range Scan of 27400 user documents took 488.168µs
2016/04/06 08:21:20 PERFSTAT Range 0.2034 seconds
2016/04/06 08:21:20 Range Scan of 27500 user documents took 204.193623ms
2016/04/06 08:21:20 PERFSTAT Range 0.0019 seconds
2016/04/06 08:21:20 Range Scan of 27500 user documents took 2.098104ms
2016/04/06 08:21:20 PERFSTAT Range 0.4377 seconds
2016/04/06 08:21:20 Range Scan of 27900 user documents took 437.694993ms
2016/04/06 08:21:21 PERFSTAT Range 0.7737 seconds
2016/04/06 08:21:21 Range Scan of 28400 user documents took 773.820104ms
2016/04/06 08:21:21 PERFSTAT Range 0.0007 seconds
2016/04/06 08:21:21 Range Scan of 28400 user documents took 745.106µs
2016/04/06 08:21:21 PERFSTAT Range 0.1142 seconds
2016/04/06 08:21:21 Range Scan of 28400 user documents took 114.260495ms
2016/04/06 08:21:21 PERFSTAT Range 0.1998 seconds
2016/04/06 08:21:21 Range Scan of 28600 user documents took 199.832464ms
2016/04/06 08:21:21 PERFSTAT Range 0.0005 seconds
2016/04/06 08:21:21 Range Scan of 28600 user documents took 534.785µs
2016/04/06 08:21:22 PERFSTAT Range 1.0541 seconds
2016/04/06 08:21:22 Range Scan of 29300 user documents took 1.055278487s
2016/04/06 08:21:22 PERFSTAT Range 0.0007 seconds
2016/04/06 08:21:22 Range Scan of 29300 user documents took 752.332µs
2016/04/06 08:21:23 PERFSTAT Range 0.3906 seconds
2016/04/06 08:21:23 Range Scan of 29600 user documents took 390.603145ms
2016/04/06 08:21:23 PERFSTAT Range 0.1523 seconds
2016/04/06 08:21:23 Range Scan of 29700 user documents took 152.374397ms
2016/04/06 08:21:23 PERFSTAT Range 0.0006 seconds
2016/04/06 08:21:23 Range Scan of 29700 user documents took 598.013µs
2016/04/06 08:21:23 PERFSTAT Range 0.0007 seconds
2016/04/06 08:21:23 Range Scan of 29700 user documents took 756.712µs
2016/04/06 08:21:24 PERFSTAT Range 0.8835 seconds
2016/04/06 08:21:24 Range Scan of 30400 user documents took 883.547778ms
2016/04/06 08:21:24 PERFSTAT Range 0.2005 seconds
2016/04/06 08:21:24 Range Scan of 30500 user documents took 200.545745ms
2016/04/06 08:21:24 PERFSTAT Range 0.0641 seconds
2016/04/06 08:21:24 Range Scan of 30600 user documents took 64.114569ms
2016/04/06 08:21:24 PERFSTAT Range 0.0005 seconds
2016/04/06 08:21:24 Range Scan of 30600 user documents took 573.366µs
2016/04/06 08:21:25 PERFSTAT Range 0.4095 seconds
2016/04/06 08:21:25 Range Scan of 30900 user documents took 409.537971ms
2016/04/06 08:21:25 PERFSTAT Range 0.0903 seconds
2016/04/06 08:21:25 Range Scan of 30900 user documents took 90.376206ms
2016/04/06 08:21:25 PERFSTAT Range 0.1304 seconds
2016/04/06 08:21:25 Range Scan of 31000 user documents took 130.46532ms
2016/04/06 08:21:25 PERFSTAT Range 0.0005 seconds
2016/04/06 08:21:25 Range Scan of 31000 user documents took 545.788µs
2016/04/06 08:21:25 PERFSTAT Range 0.0005 seconds
2016/04/06 08:21:25 Range Scan of 31000 user documents took 519.386µs
2016/04/06 08:21:25 PERFSTAT Range 0.0004 seconds
2016/04/06 08:21:25 Range Scan of 31000 user documents took 422.339µs
2016/04/06 08:21:25 PERFSTAT Range 0.4222 seconds
2016/04/06 08:21:25 Range Scan of 31300 user documents took 422.192965ms
2016/04/06 08:21:25 PERFSTAT Range 0.0006 seconds
2016/04/06 08:21:25 Range Scan of 31300 user documents took 653.589µs
2016/04/06 08:21:26 PERFSTAT Range 0.3620 seconds
2016/04/06 08:21:26 Range Scan of 31500 user documents took 362.117723ms
2016/04/06 08:21:26 PERFSTAT Range 0.5540 seconds
2016/04/06 08:21:26 Range Scan of 31900 user documents took 554.010334ms
2016/04/06 08:21:26 PERFSTAT Range 0.2884 seconds
2016/04/06 08:21:26 Range Scan of 32000 user documents took 288.444501ms
2016/04/06 08:21:27 PERFSTAT Range 0.4785 seconds
2016/04/06 08:21:27 Range Scan of 32400 user documents took 478.567211ms
2016/04/06 08:21:27 PERFSTAT Range 0.0902 seconds
2016/04/06 08:21:27 Range Scan of 32400 user documents took 90.282561ms
2016/04/06 08:21:28 PERFSTAT Range 0.9364 seconds
2016/04/06 08:21:28 Range Scan of 33000 user documents took 936.470089ms
2016/04/06 08:21:28 PERFSTAT Range 0.0006 seconds
2016/04/06 08:21:28 Range Scan of 33000 user documents took 679.971µs
2016/04/06 08:21:28 PERFSTAT Range 0.0491 seconds
2016/04/06 08:21:28 Range Scan of 33000 user documents took 49.159107ms
2016/04/06 08:21:29 PERFSTAT Range 0.8476 seconds
2016/04/06 08:21:29 Range Scan of 33600 user documents took 847.687026ms
2016/04/06 08:21:29 PERFSTAT Range 0.5263 seconds
2016/04/06 08:21:29 Range Scan of 33800 user documents took 526.349024ms
2016/04/06 08:21:29 PERFSTAT Range 0.0004 seconds
2016/04/06 08:21:29 Range Scan of 33800 user documents took 432.478µs
2016/04/06 08:21:29 PERFSTAT Range 0.0006 seconds
2016/04/06 08:21:29 Range Scan of 33800 user documents took 670.491µs
2016/04/06 08:21:29 PERFSTAT Range 0.0040 seconds
2016/04/06 08:21:29 Range Scan of 33800 user documents took 4.054027ms
2016/04/06 08:21:29 PERFSTAT Range 0.0482 seconds
2016/04/06 08:21:29 Range Scan of 33900 user documents took 48.241607ms
2016/04/06 08:21:30 PERFSTAT Range 0.8943 seconds
2016/04/06 08:21:30 Range Scan of 34400 user documents took 894.382408ms
2016/04/06 08:21:31 PERFSTAT Range 1.1181 seconds
2016/04/06 08:21:31 Range Scan of 35200 user documents took 1.11809985s
2016/04/06 08:21:31 PERFSTAT Range 0.0006 seconds
2016/04/06 08:21:31 Range Scan of 35200 user documents took 663.731µs
2016/04/06 08:21:33 PERFSTAT Range 1.1443 seconds
2016/04/06 08:21:33 Range Scan of 35900 user documents took 1.144335799s
2016/04/06 08:21:33 PERFSTAT Range 0.0007 seconds
2016/04/06 08:21:33 Range Scan of 35900 user documents took 722.062µs
2016/04/06 08:21:33 PERFSTAT Range 0.1450 seconds
2016/04/06 08:21:33 Range Scan of 36000 user documents took 144.998517ms
2016/04/06 08:21:33 PERFSTAT Range 0.8105 seconds
2016/04/06 08:21:33 Range Scan of 36500 user documents took 810.519798ms
2016/04/06 08:21:33 PERFSTAT Range 0.0007 seconds
2016/04/06 08:21:33 Range Scan of 36500 user documents took 721.108µs
2016/04/06 08:21:34 PERFSTAT Range 0.1573 seconds
2016/04/06 08:21:34 Range Scan of 36600 user documents took 157.37167ms
2016/04/06 08:21:35 PERFSTAT Range 1.1131 seconds
2016/04/06 08:21:35 Range Scan of 37300 user documents took 1.113189074s
2016/04/06 08:21:35 PERFSTAT Range 0.0005 seconds
2016/04/06 08:21:35 Range Scan of 37300 user documents took 556.475µs
2016/04/06 08:21:35 PERFSTAT Range 0.0005 seconds
2016/04/06 08:21:35 Range Scan of 37300 user documents took 503.194µs
2016/04/06 08:21:36 PERFSTAT Range 1.5114 seconds
2016/04/06 08:21:36 Range Scan of 38100 user documents took 1.511448341s
2016/04/06 08:21:36 PERFSTAT Range 0.0006 seconds
2016/04/06 08:21:36 Range Scan of 38100 user documents took 612.395µs
2016/04/06 08:21:38 PERFSTAT Range 1.2678 seconds
2016/04/06 08:21:38 Range Scan of 38800 user documents took 1.26788149s
2016/04/06 08:21:38 PERFSTAT Range 0.0007 seconds
2016/04/06 08:21:38 Range Scan of 38800 user documents took 706.906µs
2016/04/06 08:21:38 PERFSTAT Range 0.1432 seconds
2016/04/06 08:21:38 Range Scan of 38900 user documents took 143.2311ms
2016/04/06 08:21:38 PERFSTAT Range 0.3183 seconds
2016/04/06 08:21:38 Range Scan of 39000 user documents took 318.579895ms
2016/04/06 08:21:38 PERFSTAT Range 0.1737 seconds
2016/04/06 08:21:38 Range Scan of 39200 user documents took 173.751015ms
2016/04/06 08:21:39 PERFSTAT Range 0.5493 seconds
2016/04/06 08:21:39 Range Scan of 39600 user documents took 549.307843ms
2016/04/06 08:21:39 PERFSTAT Range 0.0008 seconds
2016/04/06 08:21:39 Range Scan of 39600 user documents took 808.052µs
2016/04/06 08:21:39 PERFSTAT Range 0.2187 seconds
2016/04/06 08:21:39 Range Scan of 39700 user documents took 218.787452ms
2016/04/06 08:21:39 PERFSTAT Range 0.0006 seconds
2016/04/06 08:21:39 Range Scan of 39700 user documents took 7.897954ms
2016/04/06 08:21:39 PERFSTAT Range 0.0005 seconds
2016/04/06 08:21:39 Range Scan of 39700 user documents took 521.898µs
2016/04/06 08:21:40 PERFSTAT Range 0.7895 seconds
2016/04/06 08:21:40 Range Scan of 40200 user documents took 789.550026ms
2016/04/06 08:21:40 PERFSTAT Range 0.0006 seconds
2016/04/06 08:21:40 Range Scan of 40200 user documents took 627.693µs
2016/04/06 08:21:40 PERFSTAT Range 0.0005 seconds
2016/04/06 08:21:40 Range Scan of 40200 user documents took 509.332µs
2016/04/06 08:21:40 PERFSTAT Range 0.0005 seconds
2016/04/06 08:21:40 Range Scan of 40200 user documents took 521.589µs
2016/04/06 08:21:40 PERFSTAT Range 0.0004 seconds
2016/04/06 08:21:40 Range Scan of 40200 user documents took 501.076µs
2016/04/06 08:21:40 PERFSTAT Range 0.5497 seconds
2016/04/06 08:21:40 Range Scan of 40500 user documents took 549.687749ms
2016/04/06 08:21:41 PERFSTAT Range 0.2650 seconds
2016/04/06 08:21:41 Range Scan of 40700 user documents took 265.00261ms
2016/04/06 08:21:42 PERFSTAT Range 1.0226 seconds
2016/04/06 08:21:42 Range Scan of 41300 user documents took 1.02270277s
2016/04/06 08:21:42 PERFSTAT Range 0.0005 seconds
2016/04/06 08:21:42 Range Scan of 41300 user documents took 545.868µs
2016/04/06 08:21:43 PERFSTAT Range 1.8121 seconds
2016/04/06 08:21:43 Range Scan of 42600 user documents took 1.812124984s
2016/04/06 08:21:43 PERFSTAT Range 0.0007 seconds
2016/04/06 08:21:43 Range Scan of 42600 user documents took 699.505µs
2016/04/06 08:21:43 PERFSTAT Range 0.0004 seconds
2016/04/06 08:21:43 Range Scan of 42600 user documents took 460.557µs
2016/04/06 08:21:44 PERFSTAT Range 0.5046 seconds
2016/04/06 08:21:44 Range Scan of 42900 user documents took 504.605147ms
2016/04/06 08:21:45 PERFSTAT Range 1.5612 seconds
2016/04/06 08:21:45 Range Scan of 44100 user documents took 1.561290549s
2016/04/06 08:21:45 PERFSTAT Range 0.0014 seconds
2016/04/06 08:21:45 Range Scan of 44100 user documents took 1.421895ms
2016/04/06 08:21:46 PERFSTAT Range 0.2854 seconds
2016/04/06 08:21:46 Range Scan of 44200 user documents took 285.44493ms
2016/04/06 08:21:46 PERFSTAT Range 0.0005 seconds
2016/04/06 08:21:46 Range Scan of 44200 user documents took 575.223µs
2016/04/06 08:21:46 PERFSTAT Range 0.0004 seconds
2016/04/06 08:21:46 Range Scan of 44200 user documents took 539.199µs
2016/04/06 08:21:46 PERFSTAT Range 0.2895 seconds
2016/04/06 08:21:46 Range Scan of 44400 user documents took 289.51117ms
2016/04/06 08:21:47 PERFSTAT Range 1.0212 seconds
2016/04/06 08:21:47 Range Scan of 45100 user documents took 1.021269847s
2016/04/06 08:21:47 PERFSTAT Range 0.0014 seconds
2016/04/06 08:21:47 Range Scan of 45100 user documents took 1.472057ms
2016/04/06 08:21:47 PERFSTAT Range 0.0005 seconds
2016/04/06 08:21:47 Range Scan of 45100 user documents took 537.106µs
2016/04/06 08:21:47 PERFSTAT Range 0.0032 seconds
2016/04/06 08:21:47 Range Scan of 45100 user documents took 3.243667ms
2016/04/06 08:21:47 PERFSTAT Range 0.0005 seconds
2016/04/06 08:21:47 Range Scan of 45100 user documents took 501.712µs
2016/04/06 08:21:47 PERFSTAT Range 0.0006 seconds
2016/04/06 08:21:47 Range Scan of 45100 user documents took 605.228µs
2016/04/06 08:21:47 PERFSTAT Range 0.0050 seconds
2016/04/06 08:21:47 Range Scan of 45100 user documents took 4.984076ms
2016/04/06 08:21:47 PERFSTAT Range 0.2900 seconds
2016/04/06 08:21:47 Range Scan of 45300 user documents took 289.992719ms
2016/04/06 08:21:47 PERFSTAT Range 0.0005 seconds
2016/04/06 08:21:47 Range Scan of 45300 user documents took 558.046µs
2016/04/06 08:21:48 PERFSTAT Range 0.8781 seconds
2016/04/06 08:21:48 Range Scan of 45900 user documents took 878.179802ms
2016/04/06 08:21:51 PERFSTAT Range 2.3705 seconds
2016/04/06 08:21:51 Range Scan of 47400 user documents took 2.37085304s
2016/04/06 08:21:51 PERFSTAT Range 0.0024 seconds
2016/04/06 08:21:51 Range Scan of 47400 user documents took 2.413118ms
2016/04/06 08:21:51 PERFSTAT Range 0.1592 seconds
2016/04/06 08:21:51 Range Scan of 47500 user documents took 159.243278ms
2016/04/06 08:21:51 PERFSTAT Range 0.0008 seconds
2016/04/06 08:21:51 Range Scan of 47500 user documents took 803.619µs
2016/04/06 08:21:51 PERFSTAT Range 0.4132 seconds
2016/04/06 08:21:51 Range Scan of 47700 user documents took 413.203692ms
2016/04/06 08:21:52 PERFSTAT Range 0.8060 seconds
2016/04/06 08:21:52 Range Scan of 48300 user documents took 806.073336ms
2016/04/06 08:21:52 PERFSTAT Range 0.0005 seconds
2016/04/06 08:21:52 Range Scan of 48300 user documents took 548.201µs
2016/04/06 08:21:53 PERFSTAT Range 0.9290 seconds
2016/04/06 08:21:53 Range Scan of 48900 user documents took 929.050675ms
2016/04/06 08:21:54 PERFSTAT Range 0.5973 seconds
2016/04/06 08:21:54 Range Scan of 49300 user documents took 597.377451ms
2016/04/06 08:21:55 PERFSTAT Range 1.8418 seconds
2016/04/06 08:21:55 Range Scan of 50600 user documents took 1.841858124s
2016/04/06 08:21:55 PERFSTAT Range 0.0006 seconds
2016/04/06 08:21:55 Range Scan of 50600 user documents took 599.333µs
2016/04/06 08:21:55 PERFSTAT Range 0.0006 seconds
2016/04/06 08:21:55 Range Scan of 50600 user documents took 690.398µs
2016/04/06 08:21:55 PERFSTAT Range 0.1361 seconds
2016/04/06 08:21:55 Range Scan of 50700 user documents took 136.138364ms
2016/04/06 08:21:56 PERFSTAT Range 0.0962 seconds
2016/04/06 08:21:56 Range Scan of 50800 user documents took 96.285377ms
2016/04/06 08:21:56 PERFSTAT Range 0.0005 seconds
2016/04/06 08:21:56 Range Scan of 50800 user documents took 504.892µs
2016/04/06 08:21:58 PERFSTAT Range 2.1111 seconds
2016/04/06 08:21:58 Range Scan of 52300 user documents took 2.111181688s
2016/04/06 08:21:58 PERFSTAT Range 0.0011 seconds
2016/04/06 08:21:58 Range Scan of 52300 user documents took 1.176665ms
2016/04/06 08:21:58 PERFSTAT Range 0.0821 seconds
2016/04/06 08:21:58 Range Scan of 52300 user documents took 82.175165ms
2016/04/06 08:21:58 PERFSTAT Range 0.0005 seconds
2016/04/06 08:21:58 Range Scan of 52300 user documents took 541.575µs
2016/04/06 08:21:58 PERFSTAT Range 0.0004 seconds
2016/04/06 08:21:58 Range Scan of 52300 user documents took 421.393µs
2016/04/06 08:21:58 PERFSTAT Range 0.0004 seconds
2016/04/06 08:21:58 Range Scan of 52300 user documents took 401.439µs
2016/04/06 08:21:58 PERFSTAT Range 0.0003 seconds
2016/04/06 08:21:58 Range Scan of 52300 user documents took 364.783µs
2016/04/06 08:21:58 PERFSTAT Range 0.0004 seconds
2016/04/06 08:21:58 Range Scan of 52300 user documents took 452.171µs
2016/04/06 08:21:58 PERFSTAT Range 0.4451 seconds
2016/04/06 08:21:58 Range Scan of 52600 user documents took 445.187158ms
2016/04/06 08:21:58 PERFSTAT Range 0.0005 seconds
2016/04/06 08:21:58 Range Scan of 52600 user documents took 559.129µs
2016/04/06 08:22:00 PERFSTAT Range 1.3118 seconds
2016/04/06 08:22:00 Range Scan of 53500 user documents took 1.311788804s
2016/04/06 08:22:01 PERFSTAT Range 1.0103 seconds
2016/04/06 08:22:01 Range Scan of 54100 user documents took 1.010376228s
2016/04/06 08:22:01 PERFSTAT Range 0.0028 seconds
2016/04/06 08:22:01 Range Scan of 54100 user documents took 2.875869ms
2016/04/06 08:22:02 PERFSTAT Range 1.4050 seconds
2016/04/06 08:22:02 Range Scan of 55100 user documents took 1.405087367s
2016/04/06 08:22:04 PERFSTAT Range 2.4850 seconds
2016/04/06 08:22:04 Range Scan of 56800 user documents took 2.485044909s
2016/04/06 08:22:04 PERFSTAT Range 0.0019 seconds
2016/04/06 08:22:04 Range Scan of 56800 user documents took 1.929247ms
2016/04/06 08:22:06 PERFSTAT Range 1.2452 seconds
2016/04/06 08:22:06 Range Scan of 57600 user documents took 1.245276983s
2016/04/06 08:22:06 PERFSTAT Range 0.0007 seconds
2016/04/06 08:22:06 Range Scan of 57600 user documents took 700.875µs
2016/04/06 08:22:06 PERFSTAT Range 0.0011 seconds
2016/04/06 08:22:06 Range Scan of 57600 user documents took 1.090509ms
2016/04/06 08:22:06 PERFSTAT Range 0.4141 seconds
2016/04/06 08:22:06 Range Scan of 58000 user documents took 414.137789ms
2016/04/06 08:22:06 PERFSTAT Range 0.0012 seconds
2016/04/06 08:22:06 Range Scan of 58000 user documents took 1.212874ms
2016/04/06 08:22:06 PERFSTAT Range 0.0010 seconds
2016/04/06 08:22:06 Range Scan of 58000 user documents took 1.988387ms
2016/04/06 08:22:06 PERFSTAT Range 0.0004 seconds
2016/04/06 08:22:06 Range Scan of 58000 user documents took 423.655µs
2016/04/06 08:22:07 PERFSTAT Range 0.4793 seconds
2016/04/06 08:22:07 Range Scan of 58300 user documents took 479.334395ms
2016/04/06 08:22:07 PERFSTAT Range 0.6853 seconds
2016/04/06 08:22:07 Range Scan of 58800 user documents took 685.319657ms
2016/04/06 08:22:07 PERFSTAT Range 0.0005 seconds
2016/04/06 08:22:07 Range Scan of 58800 user documents took 570.808µs
2016/04/06 08:22:07 PERFSTAT Range 0.0003 seconds
2016/04/06 08:22:07 Range Scan of 58800 user documents took 379.178µs
2016/04/06 08:22:07 PERFSTAT Range 0.0004 seconds
2016/04/06 08:22:07 Range Scan of 58800 user documents took 381.238µs
2016/04/06 08:22:07 PERFSTAT Range 0.0003 seconds
2016/04/06 08:22:07 Range Scan of 58800 user documents took 345.004µs
2016/04/06 08:22:07 PERFSTAT Range 0.0003 seconds
2016/04/06 08:22:07 Range Scan of 58800 user documents took 351.021µs
2016/04/06 08:22:08 PERFSTAT Range 1.0855 seconds
2016/04/06 08:22:08 Range Scan of 59300 user documents took 1.085631002s
2016/04/06 08:22:08 PERFSTAT Range 0.0005 seconds
2016/04/06 08:22:08 Range Scan of 59300 user documents took 543.9µs
2016/04/06 08:22:08 PERFSTAT Range 0.0006 seconds
2016/04/06 08:22:08 Range Scan of 59300 user documents took 641.986µs
2016/04/06 08:22:08 PERFSTAT Range 0.0005 seconds
2016/04/06 08:22:08 Range Scan of 59300 user documents took 499.091µs
2016/04/06 08:22:08 PERFSTAT Range 0.0013 seconds
2016/04/06 08:22:08 Range Scan of 59300 user documents took 1.305522ms
2016/04/06 08:22:09 PERFSTAT Range 0.4907 seconds
2016/04/06 08:22:09 Range Scan of 59700 user documents took 491.205476ms
2016/04/06 08:22:09 PERFSTAT Range 0.0015 seconds
2016/04/06 08:22:09 Range Scan of 59700 user documents took 1.533108ms
2016/04/06 08:22:09 PERFSTAT Range 0.0021 seconds
2016/04/06 08:22:09 Range Scan of 59700 user documents took 2.385937ms
2016/04/06 08:22:09 PERFSTAT Range 0.0014 seconds
2016/04/06 08:22:09 Range Scan of 59700 user documents took 1.468248ms
2016/04/06 08:22:12 PERFSTAT Range 2.7364 seconds
2016/04/06 08:22:12 Range Scan of 61600 user documents took 2.736427859s
2016/04/06 08:22:12 PERFSTAT Range 0.3677 seconds
2016/04/06 08:22:12 Range Scan of 61800 user documents took 367.728249ms
2016/04/06 08:22:15 PERFSTAT Range 3.3054 seconds
2016/04/06 08:22:15 Range Scan of 64100 user documents took 3.305482744s
2016/04/06 08:22:15 PERFSTAT Range 0.0006 seconds
2016/04/06 08:22:15 Range Scan of 64100 user documents took 601.913µs
2016/04/06 08:22:15 PERFSTAT Range 0.0004 seconds
2016/04/06 08:22:15 Range Scan of 64100 user documents took 471.32µs
2016/04/06 08:22:16 PERFSTAT Range 0.8202 seconds
2016/04/06 08:22:16 Range Scan of 64700 user documents took 820.270547ms
2016/04/06 08:22:16 PERFSTAT Range 0.0006 seconds
2016/04/06 08:22:16 Range Scan of 64700 user documents took 641.046µs
2016/04/06 08:22:19 PERFSTAT Range 2.4992 seconds
2016/04/06 08:22:19 Range Scan of 66400 user documents took 2.499280295s
2016/04/06 08:22:19 PERFSTAT Range 0.0015 seconds
2016/04/06 08:22:19 Range Scan of 66400 user documents took 1.546299ms
2016/04/06 08:22:19 PERFSTAT Range 0.0015 seconds
2016/04/06 08:22:19 Range Scan of 66400 user documents took 1.476693ms
2016/04/06 08:22:20 PERFSTAT Range 1.2632 seconds
2016/04/06 08:22:20 Range Scan of 67200 user documents took 1.263208836s
2016/04/06 08:22:20 PERFSTAT Range 0.0006 seconds
2016/04/06 08:22:20 Range Scan of 67200 user documents took 661.107µs
2016/04/06 08:22:20 PERFSTAT Range 0.0005 seconds
2016/04/06 08:22:20 Range Scan of 67200 user documents took 506.272µs
2016/04/06 08:22:22 PERFSTAT Range 2.1844 seconds
2016/04/06 08:22:22 Range Scan of 68600 user documents took 2.184405904s
2016/04/06 08:22:22 PERFSTAT Range 0.1179 seconds
2016/04/06 08:22:22 Range Scan of 68700 user documents took 118.018712ms
2016/04/06 08:22:22 PERFSTAT Range 0.0009 seconds
2016/04/06 08:22:22 Range Scan of 68700 user documents took 958.612µs
2016/04/06 08:22:22 PERFSTAT Range 0.3226 seconds
2016/04/06 08:22:22 Range Scan of 69000 user documents took 322.689066ms
2016/04/06 08:22:22 PERFSTAT Range 0.0006 seconds
2016/04/06 08:22:22 Range Scan of 69000 user documents took 667.789µs
2016/04/06 08:22:23 PERFSTAT Range 0.2002 seconds
2016/04/06 08:22:23 Range Scan of 69100 user documents took 200.229573ms
2016/04/06 08:22:23 PERFSTAT Range 0.0006 seconds
2016/04/06 08:22:23 Range Scan of 69100 user documents took 661.176µs
2016/04/06 08:22:23 PERFSTAT Range 0.0009 seconds
2016/04/06 08:22:23 Range Scan of 69100 user documents took 942.589µs
2016/04/06 08:22:24 PERFSTAT Range 0.9549 seconds
2016/04/06 08:22:24 Range Scan of 69700 user documents took 954.938879ms
2016/04/06 08:22:25 PERFSTAT Range 1.3425 seconds
2016/04/06 08:22:25 Range Scan of 70700 user documents took 1.342550767s
2016/04/06 08:22:25 PERFSTAT Range 0.0042 seconds
2016/04/06 08:22:25 Range Scan of 70700 user documents took 4.236335ms
2016/04/06 08:22:27 PERFSTAT Range 1.5535 seconds
2016/04/06 08:22:27 Range Scan of 71800 user documents took 1.55730981s
2016/04/06 08:22:30 PERFSTAT Range 2.9613 seconds
2016/04/06 08:22:30 Range Scan of 73700 user documents took 2.961368741s
2016/04/06 08:22:33 PERFSTAT Range 3.8531 seconds
2016/04/06 08:22:33 Range Scan of 76100 user documents took 3.853136284s
2016/04/06 08:22:33 PERFSTAT Range 0.0005 seconds
2016/04/06 08:22:33 Range Scan of 76100 user documents took 480.105µs
2016/04/06 08:22:33 PERFSTAT Range 0.0026 seconds
2016/04/06 08:22:33 Range Scan of 76100 user documents took 2.673585ms
2016/04/06 08:22:37 PERFSTAT Range 3.8828 seconds
2016/04/06 08:22:37 Range Scan of 78400 user documents took 3.893852459s
2016/04/06 08:22:37 PERFSTAT Range 0.0108 seconds
2016/04/06 08:22:37 Range Scan of 78400 user documents took 10.947748ms
2016/04/06 08:22:37 PERFSTAT Range 0.0005 seconds
2016/04/06 08:22:37 Range Scan of 78400 user documents took 571.658µs
2016/04/06 08:22:39 PERFSTAT Range 1.6547 seconds
2016/04/06 08:22:39 Range Scan of 79400 user documents took 1.6547466s
2016/04/06 08:22:39 PERFSTAT Range 0.0018 seconds
2016/04/06 08:22:39 Range Scan of 79400 user documents took 1.821446ms
2016/04/06 08:22:39 PERFSTAT Range 0.0005 seconds
2016/04/06 08:22:39 Range Scan of 79400 user documents took 485.764µs
2016/04/06 08:22:41 PERFSTAT Range 1.8925 seconds
2016/04/06 08:22:41 Range Scan of 80200 user documents took 1.892496427s
--- PASS: TestRangeWithConcurrentAddMuts (123.33s)
=== RUN   TestRangeWithConcurrentDelMuts
2016/04/06 08:22:41 In TestRangeWithConcurrentDelMuts()
2016/04/06 08:22:41 Generating JSON docs
2016/04/06 08:22:43 Setting initial JSON docs in KV
2016/04/06 08:23:04 All indexers are active
2016/04/06 08:23:04 Creating a 2i
2016/04/06 08:23:04 Index found:  index_company
2016/04/06 08:23:04 In Range Scan
2016/04/06 08:23:04 CreateDocs:: Delete mutations
2016/04/06 08:23:05 PERFSTAT Range 0.8912 seconds
2016/04/06 08:23:05 Range Scan of 29910 user documents took 891.229668ms
2016/04/06 08:23:05 PERFSTAT Range 0.0005 seconds
2016/04/06 08:23:05 Range Scan of 29910 user documents took 500.499µs
2016/04/06 08:23:06 PERFSTAT Range 1.0395 seconds
2016/04/06 08:23:06 Range Scan of 29815 user documents took 1.039486563s
2016/04/06 08:23:06 PERFSTAT Range 0.0027 seconds
2016/04/06 08:23:06 Range Scan of 29815 user documents took 2.717818ms
2016/04/06 08:23:06 PERFSTAT Range 0.0048 seconds
2016/04/06 08:23:06 Range Scan of 29815 user documents took 4.915909ms
2016/04/06 08:23:06 PERFSTAT Range 0.0004 seconds
2016/04/06 08:23:06 Range Scan of 29815 user documents took 471.817µs
2016/04/06 08:23:07 PERFSTAT Range 1.1596 seconds
2016/04/06 08:23:07 Range Scan of 29705 user documents took 1.165049457s
2016/04/06 08:23:07 PERFSTAT Range 0.2966 seconds
2016/04/06 08:23:07 Range Scan of 29680 user documents took 296.66142ms
2016/04/06 08:23:07 PERFSTAT Range 0.0027 seconds
2016/04/06 08:23:07 Range Scan of 29680 user documents took 2.700409ms
2016/04/06 08:23:08 PERFSTAT Range 1.1543 seconds
2016/04/06 08:23:08 Range Scan of 29590 user documents took 1.154312263s
2016/04/06 08:23:08 PERFSTAT Range 0.3003 seconds
2016/04/06 08:23:08 Range Scan of 29555 user documents took 300.661326ms
2016/04/06 08:23:08 PERFSTAT Range 0.0012 seconds
2016/04/06 08:23:08 Range Scan of 29555 user documents took 1.271892ms
2016/04/06 08:23:11 PERFSTAT Range 2.6219 seconds
2016/04/06 08:23:11 Range Scan of 29330 user documents took 2.621927353s
2016/04/06 08:23:11 PERFSTAT Range 0.0005 seconds
2016/04/06 08:23:11 Range Scan of 29330 user documents took 481.583µs
2016/04/06 08:23:11 PERFSTAT Range 0.0009 seconds
2016/04/06 08:23:11 Range Scan of 29330 user documents took 929.901µs
2016/04/06 08:23:11 PERFSTAT Range 0.0004 seconds
2016/04/06 08:23:11 Range Scan of 29330 user documents took 416.619µs
2016/04/06 08:23:12 PERFSTAT Range 1.3550 seconds
2016/04/06 08:23:12 Range Scan of 29210 user documents took 1.355064738s
2016/04/06 08:23:12 PERFSTAT Range 0.0007 seconds
2016/04/06 08:23:12 Range Scan of 29210 user documents took 738.543µs
2016/04/06 08:23:12 PERFSTAT Range 0.0034 seconds
2016/04/06 08:23:12 Range Scan of 29210 user documents took 3.393948ms
2016/04/06 08:23:12 PERFSTAT Range 0.0126 seconds
2016/04/06 08:23:12 Range Scan of 29210 user documents took 12.625938ms
2016/04/06 08:23:15 PERFSTAT Range 2.3431 seconds
2016/04/06 08:23:15 Range Scan of 28995 user documents took 2.343111117s
2016/04/06 08:23:16 PERFSTAT Range 1.6127 seconds
2016/04/06 08:23:16 Range Scan of 28855 user documents took 1.612978976s
2016/04/06 08:23:20 PERFSTAT Range 3.6139 seconds
2016/04/06 08:23:20 Range Scan of 28515 user documents took 3.613976279s
2016/04/06 08:23:20 PERFSTAT Range 0.0009 seconds
2016/04/06 08:23:20 Range Scan of 28515 user documents took 884.594µs
2016/04/06 08:23:22 PERFSTAT Range 2.3210 seconds
2016/04/06 08:23:22 Range Scan of 28280 user documents took 2.321074438s
2016/04/06 08:23:22 PERFSTAT Range 0.0021 seconds
2016/04/06 08:23:22 Range Scan of 28275 user documents took 2.137509ms
2016/04/06 08:23:22 PERFSTAT Range 0.0007 seconds
2016/04/06 08:23:22 Range Scan of 28275 user documents took 688.569µs
2016/04/06 08:23:22 PERFSTAT Range 0.0039 seconds
2016/04/06 08:23:22 Range Scan of 28275 user documents took 3.915645ms
2016/04/06 08:23:22 PERFSTAT Range 0.0016 seconds
2016/04/06 08:23:22 Range Scan of 28275 user documents took 1.58989ms
2016/04/06 08:23:22 PERFSTAT Range 0.0005 seconds
2016/04/06 08:23:22 Range Scan of 28275 user documents took 492.445µs
2016/04/06 08:23:23 PERFSTAT Range 1.0310 seconds
2016/04/06 08:23:23 Range Scan of 28180 user documents took 1.031078135s
2016/04/06 08:23:23 PERFSTAT Range 0.0055 seconds
2016/04/06 08:23:23 Range Scan of 28180 user documents took 5.534325ms
2016/04/06 08:23:24 PERFSTAT Range 0.8462 seconds
2016/04/06 08:23:24 Range Scan of 28110 user documents took 846.223542ms
2016/04/06 08:23:24 PERFSTAT Range 0.0005 seconds
2016/04/06 08:23:24 Range Scan of 28110 user documents took 533.968µs
2016/04/06 08:23:24 PERFSTAT Range 0.0004 seconds
2016/04/06 08:23:24 Range Scan of 28110 user documents took 445.679µs
2016/04/06 08:23:26 PERFSTAT Range 1.4966 seconds
2016/04/06 08:23:26 Range Scan of 27970 user documents took 1.497020258s
2016/04/06 08:23:26 PERFSTAT Range 0.0009 seconds
2016/04/06 08:23:26 Range Scan of 27970 user documents took 954.486µs
2016/04/06 08:23:26 PERFSTAT Range 0.5919 seconds
2016/04/06 08:23:26 Range Scan of 27920 user documents took 591.893041ms
2016/04/06 08:23:26 PERFSTAT Range 0.0038 seconds
2016/04/06 08:23:26 Range Scan of 27920 user documents took 3.849423ms
2016/04/06 08:23:27 PERFSTAT Range 0.2584 seconds
2016/04/06 08:23:27 Range Scan of 27895 user documents took 258.47973ms
2016/04/06 08:23:27 PERFSTAT Range 0.0005 seconds
2016/04/06 08:23:27 Range Scan of 27895 user documents took 507.817µs
2016/04/06 08:23:27 PERFSTAT Range 0.0028 seconds
2016/04/06 08:23:27 Range Scan of 27895 user documents took 2.815861ms
2016/04/06 08:23:28 PERFSTAT Range 0.9163 seconds
2016/04/06 08:23:28 Range Scan of 27815 user documents took 916.380992ms
2016/04/06 08:23:28 PERFSTAT Range 0.0017 seconds
2016/04/06 08:23:28 Range Scan of 27815 user documents took 1.702856ms
2016/04/06 08:23:28 PERFSTAT Range 0.0004 seconds
2016/04/06 08:23:28 Range Scan of 27815 user documents took 458.903µs
2016/04/06 08:23:28 PERFSTAT Range 0.0004 seconds
2016/04/06 08:23:28 Range Scan of 27815 user documents took 388.791µs
2016/04/06 08:23:29 PERFSTAT Range 1.1476 seconds
2016/04/06 08:23:29 Range Scan of 27720 user documents took 1.147660047s
2016/04/06 08:23:29 PERFSTAT Range 0.0005 seconds
2016/04/06 08:23:29 Range Scan of 27720 user documents took 495.94µs
2016/04/06 08:23:29 PERFSTAT Range 0.0004 seconds
2016/04/06 08:23:29 Range Scan of 27720 user documents took 437.09µs
2016/04/06 08:23:32 PERFSTAT Range 3.1407 seconds
2016/04/06 08:23:32 Range Scan of 27420 user documents took 3.140762618s
2016/04/06 08:23:32 PERFSTAT Range 0.0005 seconds
2016/04/06 08:23:32 Range Scan of 27420 user documents took 511.419µs
2016/04/06 08:23:32 PERFSTAT Range 0.0003 seconds
2016/04/06 08:23:32 Range Scan of 27420 user documents took 363.189µs
2016/04/06 08:23:32 PERFSTAT Range 0.0003 seconds
2016/04/06 08:23:32 Range Scan of 27420 user documents took 359.227µs
2016/04/06 08:23:34 PERFSTAT Range 1.9155 seconds
2016/04/06 08:23:34 Range Scan of 27235 user documents took 1.915528675s
2016/04/06 08:23:36 PERFSTAT Range 2.0140 seconds
2016/04/06 08:23:36 Range Scan of 27030 user documents took 2.014072465s
2016/04/06 08:23:36 PERFSTAT Range 0.1369 seconds
2016/04/06 08:23:36 Range Scan of 27015 user documents took 137.007576ms
2016/04/06 08:23:36 PERFSTAT Range 0.3928 seconds
2016/04/06 08:23:36 Range Scan of 26975 user documents took 392.93227ms
2016/04/06 08:23:36 PERFSTAT Range 0.0006 seconds
2016/04/06 08:23:36 Range Scan of 26975 user documents took 617.235µs
2016/04/06 08:23:36 PERFSTAT Range 0.0004 seconds
2016/04/06 08:23:36 Range Scan of 26975 user documents took 439.981µs
2016/04/06 08:23:36 PERFSTAT Range 0.1859 seconds
2016/04/06 08:23:36 Range Scan of 26960 user documents took 185.909776ms
2016/04/06 08:23:36 PERFSTAT Range 0.0099 seconds
2016/04/06 08:23:36 Range Scan of 26960 user documents took 9.930436ms
2016/04/06 08:23:39 PERFSTAT Range 2.8215 seconds
2016/04/06 08:23:39 Range Scan of 26680 user documents took 2.821582242s
2016/04/06 08:23:39 PERFSTAT Range 0.0007 seconds
2016/04/06 08:23:39 Range Scan of 26680 user documents took 719.11µs
2016/04/06 08:23:40 PERFSTAT Range 1.0924 seconds
2016/04/06 08:23:40 Range Scan of 26590 user documents took 1.092383113s
2016/04/06 08:23:44 PERFSTAT Range 3.5045 seconds
2016/04/06 08:23:44 Range Scan of 26250 user documents took 3.504607457s
2016/04/06 08:23:46 PERFSTAT Range 2.1968 seconds
2016/04/06 08:23:46 Range Scan of 26030 user documents took 2.196881127s
2016/04/06 08:23:50 PERFSTAT Range 3.4063 seconds
2016/04/06 08:23:50 Range Scan of 25735 user documents took 3.406441981s
2016/04/06 08:23:50 PERFSTAT Range 0.7758 seconds
2016/04/06 08:23:50 Range Scan of 25675 user documents took 775.881971ms
2016/04/06 08:23:51 PERFSTAT Range 0.2761 seconds
2016/04/06 08:23:51 Range Scan of 25650 user documents took 276.118323ms
2016/04/06 08:23:51 PERFSTAT Range 0.0008 seconds
2016/04/06 08:23:51 Range Scan of 25650 user documents took 815.563µs
2016/04/06 08:23:51 PERFSTAT Range 0.0036 seconds
2016/04/06 08:23:51 Range Scan of 25650 user documents took 3.60406ms
2016/04/06 08:23:51 PERFSTAT Range 0.0005 seconds
2016/04/06 08:23:51 Range Scan of 25650 user documents took 554.234µs
2016/04/06 08:23:51 PERFSTAT Range 0.0005 seconds
2016/04/06 08:23:51 Range Scan of 25650 user documents took 511.938µs
2016/04/06 08:23:53 PERFSTAT Range 2.6822 seconds
2016/04/06 08:23:53 Range Scan of 25400 user documents took 2.682883632s
2016/04/06 08:23:54 PERFSTAT Range 0.6438 seconds
2016/04/06 08:23:54 Range Scan of 25330 user documents took 643.855399ms
2016/04/06 08:23:54 PERFSTAT Range 0.0008 seconds
2016/04/06 08:23:54 Range Scan of 25330 user documents took 868.314µs
2016/04/06 08:23:54 PERFSTAT Range 0.0005 seconds
2016/04/06 08:23:54 Range Scan of 25330 user documents took 546.177µs
2016/04/06 08:23:54 PERFSTAT Range 0.0023 seconds
2016/04/06 08:23:54 Range Scan of 25330 user documents took 2.310097ms
2016/04/06 08:23:57 PERFSTAT Range 3.2644 seconds
2016/04/06 08:23:57 Range Scan of 25020 user documents took 3.264403055s
2016/04/06 08:23:57 PERFSTAT Range 0.0005 seconds
2016/04/06 08:23:57 Range Scan of 25020 user documents took 566.391µs
2016/04/06 08:23:57 PERFSTAT Range 0.0004 seconds
2016/04/06 08:23:57 Range Scan of 25020 user documents took 377.534µs
2016/04/06 08:23:57 PERFSTAT Range 0.0003 seconds
2016/04/06 08:23:57 Range Scan of 25020 user documents took 336.173µs
2016/04/06 08:24:00 PERFSTAT Range 2.3734 seconds
2016/04/06 08:24:00 Range Scan of 24790 user documents took 2.373429431s
2016/04/06 08:24:01 PERFSTAT Range 1.4350 seconds
2016/04/06 08:24:01 Range Scan of 24665 user documents took 1.435027671s
2016/04/06 08:24:01 PERFSTAT Range 0.0005 seconds
2016/04/06 08:24:01 Range Scan of 24665 user documents took 556.386µs
2016/04/06 08:24:01 PERFSTAT Range 0.0003 seconds
2016/04/06 08:24:01 Range Scan of 24665 user documents took 366.425µs
2016/04/06 08:24:01 PERFSTAT Range 0.0003 seconds
2016/04/06 08:24:01 Range Scan of 24665 user documents took 357.072µs
2016/04/06 08:24:01 PERFSTAT Range 0.1571 seconds
2016/04/06 08:24:01 Range Scan of 24650 user documents took 157.153654ms
2016/04/06 08:24:01 PERFSTAT Range 0.0005 seconds
2016/04/06 08:24:01 Range Scan of 24650 user documents took 553.72µs
2016/04/06 08:24:03 PERFSTAT Range 2.1929 seconds
2016/04/06 08:24:03 Range Scan of 24440 user documents took 2.193075001s
2016/04/06 08:24:04 PERFSTAT Range 1.0491 seconds
2016/04/06 08:24:04 Range Scan of 24370 user documents took 1.049188815s
2016/04/06 08:24:06 PERFSTAT Range 1.2718 seconds
2016/04/06 08:24:06 Range Scan of 24295 user documents took 1.27189007s
2016/04/06 08:24:06 PERFSTAT Range 0.0006 seconds
2016/04/06 08:24:06 Range Scan of 24295 user documents took 645.598µs
2016/04/06 08:24:06 PERFSTAT Range 0.0004 seconds
2016/04/06 08:24:06 Range Scan of 24295 user documents took 438.523µs
2016/04/06 08:24:09 PERFSTAT Range 2.8721 seconds
2016/04/06 08:24:09 Range Scan of 24065 user documents took 2.872170457s
2016/04/06 08:24:09 PERFSTAT Range 0.5865 seconds
2016/04/06 08:24:09 Range Scan of 24010 user documents took 586.529468ms
2016/04/06 08:24:09 PERFSTAT Range 0.0005 seconds
2016/04/06 08:24:09 Range Scan of 24010 user documents took 512.517µs
2016/04/06 08:24:11 PERFSTAT Range 1.8426 seconds
2016/04/06 08:24:11 Range Scan of 23840 user documents took 1.84260893s
2016/04/06 08:24:11 PERFSTAT Range 0.0005 seconds
2016/04/06 08:24:11 Range Scan of 23840 user documents took 505.379µs
2016/04/06 08:24:12 PERFSTAT Range 0.7260 seconds
2016/04/06 08:24:12 Range Scan of 23765 user documents took 726.026581ms
2016/04/06 08:24:12 PERFSTAT Range 0.0007 seconds
2016/04/06 08:24:12 Range Scan of 23765 user documents took 682.894µs
2016/04/06 08:24:12 PERFSTAT Range 0.0009 seconds
2016/04/06 08:24:12 Range Scan of 23765 user documents took 953.31µs
2016/04/06 08:24:12 PERFSTAT Range 0.0005 seconds
2016/04/06 08:24:12 Range Scan of 23765 user documents took 567.889µs
2016/04/06 08:24:12 PERFSTAT Range 0.6279 seconds
2016/04/06 08:24:12 Range Scan of 23710 user documents took 627.929069ms
2016/04/06 08:24:12 PERFSTAT Range 0.0029 seconds
2016/04/06 08:24:12 Range Scan of 23710 user documents took 2.903469ms
2016/04/06 08:24:13 PERFSTAT Range 0.2121 seconds
2016/04/06 08:24:13 Range Scan of 23690 user documents took 212.133019ms
2016/04/06 08:24:13 PERFSTAT Range 0.0031 seconds
2016/04/06 08:24:13 Range Scan of 23690 user documents took 3.154501ms
2016/04/06 08:24:14 PERFSTAT Range 1.9754 seconds
2016/04/06 08:24:14 Range Scan of 23525 user documents took 1.975419652s
2016/04/06 08:24:14 PERFSTAT Range 0.0006 seconds
2016/04/06 08:24:14 Range Scan of 23525 user documents took 590.729µs
2016/04/06 08:24:15 PERFSTAT Range 0.0004 seconds
2016/04/06 08:24:15 Range Scan of 23525 user documents took 468.375µs
2016/04/06 08:24:15 PERFSTAT Range 0.0004 seconds
2016/04/06 08:24:15 Range Scan of 23525 user documents took 398.544µs
2016/04/06 08:24:15 PERFSTAT Range 0.0003 seconds
2016/04/06 08:24:15 Range Scan of 23525 user documents took 371.701µs
2016/04/06 08:24:15 PERFSTAT Range 0.2822 seconds
2016/04/06 08:24:15 Range Scan of 23500 user documents took 282.242089ms
2016/04/06 08:24:15 PERFSTAT Range 0.0005 seconds
2016/04/06 08:24:15 Range Scan of 23500 user documents took 495.989µs
2016/04/06 08:24:16 PERFSTAT Range 1.6307 seconds
2016/04/06 08:24:16 Range Scan of 23360 user documents took 1.630702922s
2016/04/06 08:24:16 PERFSTAT Range 0.0007 seconds
2016/04/06 08:24:16 Range Scan of 23360 user documents took 728.453µs
2016/04/06 08:24:16 PERFSTAT Range 0.0004 seconds
2016/04/06 08:24:16 Range Scan of 23360 user documents took 1.488913ms
2016/04/06 08:24:18 PERFSTAT Range 1.6890 seconds
2016/04/06 08:24:18 Range Scan of 23195 user documents took 1.689085532s
2016/04/06 08:24:21 PERFSTAT Range 2.5802 seconds
2016/04/06 08:24:21 Range Scan of 22955 user documents took 2.58027121s
2016/04/06 08:24:21 PERFSTAT Range 0.0005 seconds
2016/04/06 08:24:21 Range Scan of 22955 user documents took 581.913µs
2016/04/06 08:24:21 PERFSTAT Range 0.0004 seconds
2016/04/06 08:24:21 Range Scan of 22955 user documents took 414.271µs
2016/04/06 08:24:21 PERFSTAT Range 0.0005 seconds
2016/04/06 08:24:21 Range Scan of 22955 user documents took 508.51µs
2016/04/06 08:24:21 PERFSTAT Range 0.0004 seconds
2016/04/06 08:24:21 Range Scan of 22955 user documents took 445.461µs
2016/04/06 08:24:21 PERFSTAT Range 0.0004 seconds
2016/04/06 08:24:21 Range Scan of 22955 user documents took 397.742µs
2016/04/06 08:24:21 PERFSTAT Range 0.2573 seconds
2016/04/06 08:24:21 Range Scan of 22935 user documents took 257.39542ms
2016/04/06 08:24:21 PERFSTAT Range 0.0005 seconds
2016/04/06 08:24:21 Range Scan of 22935 user documents took 516.276µs
2016/04/06 08:24:23 PERFSTAT Range 1.7363 seconds
2016/04/06 08:24:23 Range Scan of 22770 user documents took 1.736607526s
2016/04/06 08:24:23 PERFSTAT Range 0.0005 seconds
2016/04/06 08:24:23 Range Scan of 22770 user documents took 2.088659ms
2016/04/06 08:24:23 PERFSTAT Range 0.1960 seconds
2016/04/06 08:24:23 Range Scan of 22750 user documents took 196.077275ms
2016/04/06 08:24:23 PERFSTAT Range 0.5098 seconds
2016/04/06 08:24:23 Range Scan of 22705 user documents took 509.887056ms
2016/04/06 08:24:23 PERFSTAT Range 0.0005 seconds
2016/04/06 08:24:23 Range Scan of 22705 user documents took 506.817µs
2016/04/06 08:24:27 PERFSTAT Range 3.4643 seconds
2016/04/06 08:24:27 Range Scan of 22400 user documents took 3.464311099s
2016/04/06 08:24:27 PERFSTAT Range 0.0012 seconds
2016/04/06 08:24:27 Range Scan of 22400 user documents took 1.278671ms
2016/04/06 08:24:27 PERFSTAT Range 0.0004 seconds
2016/04/06 08:24:27 Range Scan of 22400 user documents took 376.28µs
2016/04/06 08:24:29 PERFSTAT Range 2.0757 seconds
2016/04/06 08:24:29 Range Scan of 22230 user documents took 2.07578987s
2016/04/06 08:24:31 PERFSTAT Range 1.6182 seconds
2016/04/06 08:24:31 Range Scan of 22105 user documents took 1.618290761s
2016/04/06 08:24:32 PERFSTAT Range 1.0827 seconds
2016/04/06 08:24:32 Range Scan of 21990 user documents took 1.082710828s
2016/04/06 08:24:32 PERFSTAT Range 0.0031 seconds
2016/04/06 08:24:32 Range Scan of 21990 user documents took 3.16947ms
2016/04/06 08:24:34 PERFSTAT Range 2.1818 seconds
2016/04/06 08:24:34 Range Scan of 21810 user documents took 2.181929126s
2016/04/06 08:24:37 PERFSTAT Range 3.0622 seconds
2016/04/06 08:24:37 Range Scan of 21515 user documents took 3.062296412s
2016/04/06 08:24:38 PERFSTAT Range 1.0835 seconds
2016/04/06 08:24:38 Range Scan of 21420 user documents took 1.083516894s
2016/04/06 08:24:39 PERFSTAT Range 0.8399 seconds
2016/04/06 08:24:39 Range Scan of 21350 user documents took 839.973162ms
2016/04/06 08:24:39 PERFSTAT Range 0.0005 seconds
2016/04/06 08:24:39 Range Scan of 21350 user documents took 567.984µs
2016/04/06 08:24:41 PERFSTAT Range 1.8871 seconds
2016/04/06 08:24:41 Range Scan of 21175 user documents took 1.88710034s
2016/04/06 08:24:41 PERFSTAT Range 0.0005 seconds
2016/04/06 08:24:41 Range Scan of 21175 user documents took 523.856µs
2016/04/06 08:24:41 PERFSTAT Range 0.0004 seconds
2016/04/06 08:24:41 Range Scan of 21175 user documents took 412.425µs
2016/04/06 08:24:41 PERFSTAT Range 0.2391 seconds
2016/04/06 08:24:41 Range Scan of 21150 user documents took 239.164178ms
2016/04/06 08:24:41 PERFSTAT Range 0.0006 seconds
2016/04/06 08:24:41 Range Scan of 21150 user documents took 638.092µs
2016/04/06 08:24:42 PERFSTAT Range 1.0488 seconds
2016/04/06 08:24:42 Range Scan of 21050 user documents took 1.048807664s
2016/04/06 08:24:42 PERFSTAT Range 0.1369 seconds
2016/04/06 08:24:42 Range Scan of 21040 user documents took 137.426297ms
2016/04/06 08:24:42 PERFSTAT Range 0.0055 seconds
2016/04/06 08:24:42 Range Scan of 21040 user documents took 5.55657ms
2016/04/06 08:24:42 PERFSTAT Range 0.0028 seconds
2016/04/06 08:24:42 Range Scan of 21040 user documents took 2.847028ms
2016/04/06 08:24:45 PERFSTAT Range 3.0809 seconds
2016/04/06 08:24:45 Range Scan of 20760 user documents took 3.081003472s
2016/04/06 08:24:45 PERFSTAT Range 0.0011 seconds
2016/04/06 08:24:45 Range Scan of 20760 user documents took 1.14702ms
2016/04/06 08:24:48 PERFSTAT Range 2.6900 seconds
2016/04/06 08:24:48 Range Scan of 20480 user documents took 2.690070427s
2016/04/06 08:24:48 PERFSTAT Range 0.0008 seconds
2016/04/06 08:24:48 Range Scan of 20480 user documents took 791.488µs
2016/04/06 08:24:48 PERFSTAT Range 0.0008 seconds
2016/04/06 08:24:48 Range Scan of 20480 user documents took 953.712µs
2016/04/06 08:24:49 PERFSTAT Range 1.4109 seconds
2016/04/06 08:24:49 Range Scan of 20365 user documents took 1.411033428s
2016/04/06 08:24:49 PERFSTAT Range 0.0061 seconds
2016/04/06 08:24:49 Range Scan of 20365 user documents took 6.124192ms
2016/04/06 08:24:50 PERFSTAT Range 0.9018 seconds
2016/04/06 08:24:50 Range Scan of 20295 user documents took 901.841349ms
2016/04/06 08:24:50 PERFSTAT Range 0.0006 seconds
2016/04/06 08:24:50 Range Scan of 20295 user documents took 650.378µs
2016/04/06 08:24:50 PERFSTAT Range 0.0005 seconds
2016/04/06 08:24:50 Range Scan of 20295 user documents took 553.301µs
2016/04/06 08:24:52 PERFSTAT Range 1.9491 seconds
2016/04/06 08:24:52 Range Scan of 20125 user documents took 1.949137963s
2016/04/06 08:24:52 PERFSTAT Range 0.0005 seconds
2016/04/06 08:24:52 Range Scan of 20125 user documents took 500.272µs
2016/04/06 08:24:52 PERFSTAT Range 0.0011 seconds
2016/04/06 08:24:52 Range Scan of 20125 user documents took 1.130897ms
2016/04/06 08:24:52 PERFSTAT Range 0.0003 seconds
2016/04/06 08:24:52 Range Scan of 20125 user documents took 371.78µs
2016/04/06 08:24:52 PERFSTAT Range 0.0011 seconds
2016/04/06 08:24:52 Range Scan of 20125 user documents took 1.128727ms
2016/04/06 08:24:52 PERFSTAT Range 0.0003 seconds
2016/04/06 08:24:52 Range Scan of 20125 user documents took 381.208µs
2016/04/06 08:24:52 PERFSTAT Range 0.0017 seconds
2016/04/06 08:24:52 Range Scan of 20120 user documents took 1.775688ms
2016/04/06 08:24:52 PERFSTAT Range 0.0006 seconds
2016/04/06 08:24:52 Range Scan of 20120 user documents took 1.041052ms
2016/04/06 08:24:52 PERFSTAT Range 0.0007 seconds
2016/04/06 08:24:52 Range Scan of 20120 user documents took 783.375µs
2016/04/06 08:24:53 PERFSTAT Range 0.4943 seconds
2016/04/06 08:24:53 Range Scan of 20080 user documents took 494.320304ms
2016/04/06 08:24:54 PERFSTAT Range 1.1244 seconds
2016/04/06 08:24:54 Range Scan of 19995 user documents took 1.124405754s
2016/04/06 08:24:54 PERFSTAT Range 0.0012 seconds
2016/04/06 08:24:54 Range Scan of 19995 user documents took 1.234161ms
2016/04/06 08:24:54 PERFSTAT Range 0.0007 seconds
2016/04/06 08:24:54 Range Scan of 19995 user documents took 690.859µs
2016/04/06 08:24:54 PERFSTAT Range 0.0008 seconds
2016/04/06 08:24:54 Range Scan of 19995 user documents took 807.428µs
2016/04/06 08:24:54 PERFSTAT Range 0.0005 seconds
2016/04/06 08:24:54 Range Scan of 19995 user documents took 507.794µs
2016/04/06 08:24:54 PERFSTAT Range 0.0007 seconds
2016/04/06 08:24:54 Range Scan of 19995 user documents took 702.254µs
2016/04/06 08:24:55 PERFSTAT Range 1.0839 seconds
2016/04/06 08:24:55 Range Scan of 19885 user documents took 1.083962763s
2016/04/06 08:24:56 PERFSTAT Range 1.1123 seconds
2016/04/06 08:24:56 Range Scan of 19785 user documents took 1.112434838s
2016/04/06 08:24:56 PERFSTAT Range 0.0005 seconds
2016/04/06 08:24:56 Range Scan of 19785 user documents took 495.159µs
2016/04/06 08:24:58 PERFSTAT Range 2.0099 seconds
2016/04/06 08:24:58 Range Scan of 19610 user documents took 2.010188263s
2016/04/06 08:24:59 PERFSTAT Range 0.8235 seconds
2016/04/06 08:24:59 Range Scan of 19530 user documents took 823.531834ms
2016/04/06 08:24:59 PERFSTAT Range 0.0945 seconds
2016/04/06 08:24:59 Range Scan of 19520 user documents took 94.620108ms
2016/04/06 08:25:00 PERFSTAT Range 1.1012 seconds
2016/04/06 08:25:00 Range Scan of 19425 user documents took 1.101304972s
2016/04/06 08:25:01 PERFSTAT Range 1.0684 seconds
2016/04/06 08:25:01 Range Scan of 19330 user documents took 1.068462648s
2016/04/06 08:25:01 PERFSTAT Range 0.0017 seconds
2016/04/06 08:25:01 Range Scan of 19330 user documents took 1.764375ms
2016/04/06 08:25:01 PERFSTAT Range 0.0004 seconds
2016/04/06 08:25:01 Range Scan of 19330 user documents took 457.147µs
2016/04/06 08:25:01 PERFSTAT Range 0.0015 seconds
2016/04/06 08:25:01 Range Scan of 19330 user documents took 1.493629ms
2016/04/06 08:25:01 PERFSTAT Range 0.0004 seconds
2016/04/06 08:25:01 Range Scan of 19330 user documents took 464.477µs
2016/04/06 08:25:01 PERFSTAT Range 0.0007 seconds
2016/04/06 08:25:01 Range Scan of 19330 user documents took 773.052µs
2016/04/06 08:25:02 PERFSTAT Range 1.0321 seconds
2016/04/06 08:25:02 Range Scan of 19240 user documents took 1.032138017s
2016/04/06 08:25:02 PERFSTAT Range 0.1679 seconds
2016/04/06 08:25:02 Range Scan of 19225 user documents took 168.190768ms
2016/04/06 08:25:02 PERFSTAT Range 0.0005 seconds
2016/04/06 08:25:02 Range Scan of 19225 user documents took 580.495µs
2016/04/06 08:25:04 PERFSTAT Range 2.1034 seconds
2016/04/06 08:25:04 Range Scan of 19090 user documents took 2.103456123s
--- PASS: TestRangeWithConcurrentDelMuts (143.57s)
=== RUN   TestScanWithConcurrentIndexOps
2016/04/06 08:25:04 In TestScanWithConcurrentIndexOps()
2016/04/06 08:25:04 Generating JSON docs
2016/04/06 08:25:12 Setting initial JSON docs in KV
2016/04/06 08:26:20 All indexers are active
2016/04/06 08:26:20 Creating a 2i
2016/04/06 08:26:20 Index found:  index_company
2016/04/06 08:26:20 In Range Scan
2016/04/06 08:26:20 Create and Drop index operations
2016/04/06 08:26:20 PERFSTAT Range 0.0022 seconds
2016/04/06 08:26:20 Range Scan of 100000 user documents took 2.189427ms
2016/04/06 08:26:20 PERFSTAT Range 0.0009 seconds
2016/04/06 08:26:20 Range Scan of 100000 user documents took 968.425µs
2016/04/06 08:26:34 PERFSTAT Range 13.4650 seconds
2016/04/06 08:26:34 Range Scan of 100000 user documents took 13.465068297s
2016/04/06 08:26:34 PERFSTAT Range 0.0006 seconds
2016/04/06 08:26:34 Range Scan of 100000 user documents took 618.009µs
2016/04/06 08:26:34 PERFSTAT Range 0.0005 seconds
2016/04/06 08:26:34 Range Scan of 100000 user documents took 494.136µs
2016/04/06 08:26:34 PERFSTAT Range 0.0004 seconds
2016/04/06 08:26:34 Range Scan of 100000 user documents took 461.432µs
2016/04/06 08:26:34 PERFSTAT Range 0.0003 seconds
2016/04/06 08:26:34 Range Scan of 100000 user documents took 369.724µs
2016/04/06 08:26:34 PERFSTAT Range 0.0003 seconds
2016/04/06 08:26:34 Range Scan of 100000 user documents took 360.106µs
2016/04/06 08:26:34 PERFSTAT Range 0.2921 seconds
2016/04/06 08:26:34 Range Scan of 100000 user documents took 292.117866ms
2016/04/06 08:26:34 PERFSTAT Range 0.3017 seconds
2016/04/06 08:26:34 Range Scan of 100000 user documents took 301.710652ms
2016/04/06 08:26:36 PERFSTAT Range 1.4031 seconds
2016/04/06 08:26:36 Range Scan of 100000 user documents took 1.40316121s
2016/04/06 08:26:36 PERFSTAT Range 0.0005 seconds
2016/04/06 08:26:36 Range Scan of 100000 user documents took 564.38µs
2016/04/06 08:26:36 PERFSTAT Range 0.0004 seconds
2016/04/06 08:26:36 Range Scan of 100000 user documents took 407.05µs
2016/04/06 08:26:36 PERFSTAT Range 0.0004 seconds
2016/04/06 08:26:36 Range Scan of 100000 user documents took 505.611µs
2016/04/06 08:26:37 PERFSTAT Range 1.4555 seconds
2016/04/06 08:26:37 Range Scan of 100000 user documents took 1.455511274s
2016/04/06 08:26:41 PERFSTAT Range 3.2211 seconds
2016/04/06 08:26:41 Range Scan of 100000 user documents took 3.221190014s
2016/04/06 08:26:43 PERFSTAT Range 2.4223 seconds
2016/04/06 08:26:43 Range Scan of 100000 user documents took 2.422465624s
2016/04/06 08:26:45 PERFSTAT Range 2.2396 seconds
2016/04/06 08:26:45 Range Scan of 100000 user documents took 2.239713014s
2016/04/06 08:26:45 PERFSTAT Range 0.0005 seconds
2016/04/06 08:26:45 Range Scan of 100000 user documents took 506.573µs
2016/04/06 08:26:45 PERFSTAT Range 0.0635 seconds
2016/04/06 08:26:45 Range Scan of 100000 user documents took 63.53945ms
2016/04/06 08:26:45 PERFSTAT Range 0.0004 seconds
2016/04/06 08:26:45 Range Scan of 100000 user documents took 485.59µs
2016/04/06 08:26:46 PERFSTAT Range 0.7441 seconds
2016/04/06 08:26:46 Range Scan of 100000 user documents took 744.147845ms
2016/04/06 08:26:46 PERFSTAT Range 0.0005 seconds
2016/04/06 08:26:46 Range Scan of 100000 user documents took 494.709µs
2016/04/06 08:26:47 PERFSTAT Range 1.0292 seconds
2016/04/06 08:26:47 Range Scan of 100000 user documents took 1.029286984s
2016/04/06 08:26:47 PERFSTAT Range 0.0005 seconds
2016/04/06 08:26:47 Range Scan of 100000 user documents took 538.832µs
2016/04/06 08:26:48 PERFSTAT Range 0.5070 seconds
2016/04/06 08:26:48 Range Scan of 100000 user documents took 507.038471ms
2016/04/06 08:26:48 PERFSTAT Range 0.6760 seconds
2016/04/06 08:26:48 Range Scan of 100000 user documents took 676.029756ms
2016/04/06 08:26:49 PERFSTAT Range 0.4010 seconds
2016/04/06 08:26:49 Range Scan of 100000 user documents took 401.046942ms
2016/04/06 08:26:49 PERFSTAT Range 0.0004 seconds
2016/04/06 08:26:49 Range Scan of 100000 user documents took 464.731µs
2016/04/06 08:26:49 PERFSTAT Range 0.0004 seconds
2016/04/06 08:26:49 Range Scan of 100000 user documents took 381.534µs
2016/04/06 08:26:49 PERFSTAT Range 0.0005 seconds
2016/04/06 08:26:49 Range Scan of 100000 user documents took 494.658µs
2016/04/06 08:26:49 PERFSTAT Range 0.0003 seconds
2016/04/06 08:26:49 Range Scan of 100000 user documents took 380.131µs
2016/04/06 08:26:50 PERFSTAT Range 1.3941 seconds
2016/04/06 08:26:50 Range Scan of 100000 user documents took 1.39416706s
2016/04/06 08:26:52 PERFSTAT Range 1.6804 seconds
2016/04/06 08:26:52 Range Scan of 100000 user documents took 1.680582518s
2016/04/06 08:26:52 PERFSTAT Range 0.5753 seconds
2016/04/06 08:26:52 Range Scan of 100000 user documents took 575.454349ms
2016/04/06 08:26:52 PERFSTAT Range 0.0008 seconds
2016/04/06 08:26:52 Range Scan of 100000 user documents took 872.968µs
2016/04/06 08:26:52 Created the secondary index index_age. Waiting for it become active
2016/04/06 08:26:52 Index state is INDEX_STATE_ACTIVE
2016/04/06 08:26:52 Index is now active
2016/04/06 08:26:52 PERFSTAT CreateAndBuildIndex 31.9348 seconds
2016/04/06 08:26:53 PERFSTAT Range 0.9162 seconds
2016/04/06 08:26:53 Range Scan of 100000 user documents took 916.201717ms
2016/04/06 08:26:53 PERFSTAT Range 0.0013 seconds
2016/04/06 08:26:53 Range Scan of 100000 user documents took 1.340435ms
2016/04/06 08:26:54 PERFSTAT Range 0.8014 seconds
2016/04/06 08:26:54 Range Scan of 100000 user documents took 801.54691ms
2016/04/06 08:26:54 PERFSTAT Range 0.0005 seconds
2016/04/06 08:26:54 Range Scan of 100000 user documents took 502.15µs
2016/04/06 08:26:54 PERFSTAT Range 0.0004 seconds
2016/04/06 08:26:54 Range Scan of 100000 user documents took 445.267µs
2016/04/06 08:26:54 PERFSTAT Range 0.0004 seconds
2016/04/06 08:26:54 Range Scan of 100000 user documents took 426.319µs
2016/04/06 08:26:55 PERFSTAT Range 0.9299 seconds
2016/04/06 08:26:55 Range Scan of 100000 user documents took 929.925317ms
2016/04/06 08:26:56 PERFSTAT Range 1.0395 seconds
2016/04/06 08:26:56 Range Scan of 100000 user documents took 1.039500939s
2016/04/06 08:26:56 PERFSTAT Range 2.9208 seconds
2016/04/06 08:26:57 PERFSTAT Range 1.4611 seconds
2016/04/06 08:26:57 Range Scan of 100000 user documents took 1.461138012s
2016/04/06 08:27:07 PERFSTAT Range 9.6964 seconds
2016/04/06 08:27:07 Range Scan of 100000 user documents took 9.696469262s
2016/04/06 08:27:07 PERFSTAT Range 0.0004 seconds
2016/04/06 08:27:07 Range Scan of 100000 user documents took 474.698µs
2016/04/06 08:27:07 PERFSTAT Range 0.1640 seconds
2016/04/06 08:27:07 Range Scan of 100000 user documents took 164.059166ms
2016/04/06 08:27:07 PERFSTAT Range 0.0005 seconds
2016/04/06 08:27:07 Range Scan of 100000 user documents took 515.57µs
2016/04/06 08:27:07 PERFSTAT Range 0.0004 seconds
2016/04/06 08:27:07 Range Scan of 100000 user documents took 429.808µs
2016/04/06 08:27:07 PERFSTAT Range 0.1095 seconds
2016/04/06 08:27:07 Range Scan of 100000 user documents took 109.499217ms
2016/04/06 08:27:07 PERFSTAT Range 0.0005 seconds
2016/04/06 08:27:07 Range Scan of 100000 user documents took 484.6µs
2016/04/06 08:27:09 PERFSTAT Range 1.6700 seconds
2016/04/06 08:27:09 Range Scan of 100000 user documents took 1.670067769s
2016/04/06 08:27:09 PERFSTAT Range 0.0005 seconds
2016/04/06 08:27:09 Range Scan of 100000 user documents took 489.332µs
2016/04/06 08:27:10 PERFSTAT Range 1.2580 seconds
2016/04/06 08:27:10 Range Scan of 100000 user documents took 1.258066253s
2016/04/06 08:27:11 PERFSTAT Range 0.5444 seconds
2016/04/06 08:27:11 Range Scan of 100000 user documents took 544.564024ms
2016/04/06 08:27:11 PERFSTAT Range 0.0021 seconds
2016/04/06 08:27:11 Range Scan of 100000 user documents took 2.188487ms
2016/04/06 08:27:11 PERFSTAT Range 0.0005 seconds
2016/04/06 08:27:11 Range Scan of 100000 user documents took 517.971µs
2016/04/06 08:27:11 PERFSTAT Range 0.3637 seconds
2016/04/06 08:27:11 Range Scan of 100000 user documents took 363.700652ms
2016/04/06 08:27:14 PERFSTAT Range 2.7554 seconds
2016/04/06 08:27:14 Range Scan of 100000 user documents took 2.755475243s
2016/04/06 08:27:15 PERFSTAT Range 1.0792 seconds
2016/04/06 08:27:15 Range Scan of 100000 user documents took 1.079305218s
2016/04/06 08:27:17 PERFSTAT Range 2.1999 seconds
2016/04/06 08:27:17 Range Scan of 100000 user documents took 2.199932679s
2016/04/06 08:27:17 PERFSTAT Range 0.0005 seconds
2016/04/06 08:27:17 Range Scan of 100000 user documents took 531.08µs
2016/04/06 08:27:17 PERFSTAT Range 0.1571 seconds
2016/04/06 08:27:17 Range Scan of 100000 user documents took 157.145696ms
2016/04/06 08:27:19 PERFSTAT Range 1.7438 seconds
2016/04/06 08:27:19 Range Scan of 100000 user documents took 1.743813191s
2016/04/06 08:27:19 PERFSTAT Range 0.0006 seconds
2016/04/06 08:27:19 Range Scan of 100000 user documents took 605.78µs
2016/04/06 08:27:22 PERFSTAT Range 2.9551 seconds
2016/04/06 08:27:22 Range Scan of 100000 user documents took 2.955189134s
2016/04/06 08:27:22 PERFSTAT Range 0.0004 seconds
2016/04/06 08:27:22 Range Scan of 100000 user documents took 459.292µs
2016/04/06 08:27:23 PERFSTAT Range 0.9288 seconds
2016/04/06 08:27:23 Range Scan of 100000 user documents took 928.872346ms
2016/04/06 08:27:25 PERFSTAT Range 2.1078 seconds
2016/04/06 08:27:25 Range Scan of 100000 user documents took 2.107824928s
2016/04/06 08:27:25 PERFSTAT Range 0.0005 seconds
2016/04/06 08:27:25 Range Scan of 100000 user documents took 557.995µs
2016/04/06 08:27:26 Created the secondary index index_firstname. Waiting for it become active
2016/04/06 08:27:26 Index state is INDEX_STATE_ACTIVE
2016/04/06 08:27:26 Index is now active
2016/04/06 08:27:26 PERFSTAT CreateAndBuildIndex 29.8565 seconds
2016/04/06 08:27:26 PERFSTAT Range 1.1901 seconds
2016/04/06 08:27:26 Range Scan of 100000 user documents took 1.190192602s
2016/04/06 08:27:27 PERFSTAT Range 1.1089 seconds
2016/04/06 08:27:27 Range Scan of 100000 user documents took 1.108928798s
2016/04/06 08:27:27 PERFSTAT Range 0.0005 seconds
2016/04/06 08:27:27 Range Scan of 100000 user documents took 557.595µs
2016/04/06 08:27:29 PERFSTAT Range 1.9638 seconds
2016/04/06 08:27:29 Dropping the secondary index index_age
2016/04/06 08:27:29 Index dropped
2016/04/06 08:27:29 PERFSTAT DropIndex 0.0407 seconds
2016/04/06 08:27:29 PERFSTAT Range 1.7888 seconds
2016/04/06 08:27:29 Range Scan of 100000 user documents took 1.788829913s
2016/04/06 08:27:29 PERFSTAT Range 0.0005 seconds
2016/04/06 08:27:29 Range Scan of 100000 user documents took 494.904µs
2016/04/06 08:27:30 PERFSTAT Range 0.2596 seconds
2016/04/06 08:27:30 Range Scan of 100000 user documents took 259.590569ms
2016/04/06 08:27:30 Dropping the secondary index index_firstname
2016/04/06 08:27:30 Index dropped
2016/04/06 08:27:30 PERFSTAT DropIndex 0.0385 seconds
2016/04/06 08:27:31 PERFSTAT Range 1.1906 seconds
2016/04/06 08:27:31 Range Scan of 100000 user documents took 1.190651992s
2016/04/06 08:27:31 PERFSTAT Range 0.0005 seconds
2016/04/06 08:27:31 Range Scan of 100000 user documents took 520.319µs
2016/04/06 08:27:34 PERFSTAT Range 2.9284 seconds
2016/04/06 08:27:34 Range Scan of 100000 user documents took 2.928464006s
2016/04/06 08:27:34 PERFSTAT Range 0.0023 seconds
2016/04/06 08:27:34 Range Scan of 100000 user documents took 2.528889ms
2016/04/06 08:27:39 PERFSTAT Range 5.4244 seconds
2016/04/06 08:27:39 Range Scan of 100000 user documents took 5.424663176s
2016/04/06 08:27:39 PERFSTAT Range 0.0005 seconds
2016/04/06 08:27:39 Range Scan of 100000 user documents took 543.84µs
2016/04/06 08:27:39 PERFSTAT Range 0.0005 seconds
2016/04/06 08:27:39 Range Scan of 100000 user documents took 555.349µs
2016/04/06 08:27:40 PERFSTAT Range 1.1087 seconds
2016/04/06 08:27:40 Range Scan of 100000 user documents took 1.108721121s
2016/04/06 08:27:40 PERFSTAT Range 0.0006 seconds
2016/04/06 08:27:40 Range Scan of 100000 user documents took 613.33µs
2016/04/06 08:27:40 PERFSTAT Range 0.0011 seconds
2016/04/06 08:27:40 Range Scan of 100000 user documents took 1.124727ms
2016/04/06 08:27:40 PERFSTAT Range 0.0004 seconds
2016/04/06 08:27:40 Range Scan of 100000 user documents took 443.29µs
2016/04/06 08:27:40 PERFSTAT Range 0.0004 seconds
2016/04/06 08:27:40 Range Scan of 100000 user documents took 406.983µs
2016/04/06 08:27:42 PERFSTAT Range 1.3699 seconds
2016/04/06 08:27:42 Range Scan of 100000 user documents took 1.369994125s
2016/04/06 08:27:42 PERFSTAT Range 0.0005 seconds
2016/04/06 08:27:42 Range Scan of 100000 user documents took 515.302µs
2016/04/06 08:27:42 PERFSTAT Range 0.9180 seconds
2016/04/06 08:27:42 Range Scan of 100000 user documents took 918.092506ms
2016/04/06 08:27:42 PERFSTAT Range 0.0005 seconds
2016/04/06 08:27:42 Range Scan of 100000 user documents took 539.305µs
2016/04/06 08:27:42 PERFSTAT Range 0.0004 seconds
2016/04/06 08:27:42 Range Scan of 100000 user documents took 391.742µs
2016/04/06 08:27:42 PERFSTAT Range 0.0004 seconds
2016/04/06 08:27:42 Range Scan of 100000 user documents took 380.467µs
2016/04/06 08:27:43 PERFSTAT Range 0.4158 seconds
2016/04/06 08:27:43 Range Scan of 100000 user documents took 415.852734ms
2016/04/06 08:27:43 PERFSTAT Range 0.0005 seconds
2016/04/06 08:27:43 Range Scan of 100000 user documents took 571.897µs
2016/04/06 08:27:43 PERFSTAT Range 0.0004 seconds
2016/04/06 08:27:43 Range Scan of 100000 user documents took 458.68µs
2016/04/06 08:27:43 PERFSTAT Range 0.0003 seconds
2016/04/06 08:27:43 Range Scan of 100000 user documents took 365.847µs
2016/04/06 08:27:43 PERFSTAT Range 0.0003 seconds
2016/04/06 08:27:43 Range Scan of 100000 user documents took 378.026µs
2016/04/06 08:27:43 PERFSTAT Range 0.0005 seconds
2016/04/06 08:27:43 Range Scan of 100000 user documents took 486.61µs
2016/04/06 08:27:43 PERFSTAT Range 0.5865 seconds
2016/04/06 08:27:43 Range Scan of 100000 user documents took 586.502798ms
2016/04/06 08:27:43 PERFSTAT Range 0.0004 seconds
2016/04/06 08:27:43 Range Scan of 100000 user documents took 482.272µs
2016/04/06 08:27:43 PERFSTAT Range 0.0004 seconds
2016/04/06 08:27:43 Range Scan of 100000 user documents took 380.98µs
2016/04/06 08:27:43 PERFSTAT Range 0.0003 seconds
2016/04/06 08:27:43 Range Scan of 100000 user documents took 370.524µs
2016/04/06 08:27:43 PERFSTAT Range 0.0004 seconds
2016/04/06 08:27:43 Range Scan of 100000 user documents took 377.779µs
2016/04/06 08:27:45 PERFSTAT Range 1.1521 seconds
2016/04/06 08:27:45 Range Scan of 100000 user documents took 1.152117593s
2016/04/06 08:27:45 PERFSTAT Range 0.1006 seconds
2016/04/06 08:27:45 Range Scan of 100000 user documents took 100.698892ms
2016/04/06 08:27:46 PERFSTAT Range 1.1946 seconds
2016/04/06 08:27:46 Range Scan of 100000 user documents took 1.194699193s
2016/04/06 08:27:47 PERFSTAT Range 0.7384 seconds
2016/04/06 08:27:47 Range Scan of 100000 user documents took 754.798451ms
2016/04/06 08:27:47 PERFSTAT Range 0.0007 seconds
2016/04/06 08:27:47 Range Scan of 100000 user documents took 721.445µs
2016/04/06 08:27:47 PERFSTAT Range 0.0004 seconds
2016/04/06 08:27:47 Range Scan of 100000 user documents took 409.565µs
2016/04/06 08:27:47 PERFSTAT Range 0.0003 seconds
2016/04/06 08:27:47 Range Scan of 100000 user documents took 348.27µs
2016/04/06 08:27:48 PERFSTAT Range 1.0656 seconds
2016/04/06 08:27:48 Range Scan of 100000 user documents took 1.06563052s
2016/04/06 08:27:48 PERFSTAT Range 0.0004 seconds
2016/04/06 08:27:48 Range Scan of 100000 user documents took 445.226µs
2016/04/06 08:27:48 PERFSTAT Range 0.0003 seconds
2016/04/06 08:27:48 Range Scan of 100000 user documents took 374.197µs
2016/04/06 08:27:48 PERFSTAT Range 0.0003 seconds
2016/04/06 08:27:48 Range Scan of 100000 user documents took 375.013µs
2016/04/06 08:27:49 PERFSTAT Range 1.6527 seconds
2016/04/06 08:27:49 Range Scan of 100000 user documents took 1.652795827s
2016/04/06 08:27:49 PERFSTAT Range 0.0004 seconds
2016/04/06 08:27:49 Range Scan of 100000 user documents took 465.745µs
2016/04/06 08:27:49 PERFSTAT Range 0.0004 seconds
2016/04/06 08:27:49 Range Scan of 100000 user documents took 404.39µs
2016/04/06 08:27:49 PERFSTAT Range 0.0003 seconds
2016/04/06 08:27:49 Range Scan of 100000 user documents took 335.887µs
2016/04/06 08:27:49 PERFSTAT Range 0.0003 seconds
2016/04/06 08:27:49 Range Scan of 100000 user documents took 361.75µs
2016/04/06 08:27:49 PERFSTAT Range 0.0004 seconds
2016/04/06 08:27:49 Range Scan of 100000 user documents took 404.844µs
2016/04/06 08:27:50 PERFSTAT Range 0.6642 seconds
2016/04/06 08:27:50 Range Scan of 100000 user documents took 664.3285ms
2016/04/06 08:27:50 PERFSTAT Range 0.0004 seconds
2016/04/06 08:27:50 Range Scan of 100000 user documents took 461.443µs
2016/04/06 08:27:50 PERFSTAT Range 0.0003 seconds
2016/04/06 08:27:50 Range Scan of 100000 user documents took 368.688µs
2016/04/06 08:27:50 PERFSTAT Range 0.0003 seconds
2016/04/06 08:27:50 Range Scan of 100000 user documents took 335.099µs
2016/04/06 08:27:50 PERFSTAT Range 0.0003 seconds
2016/04/06 08:27:50 Range Scan of 100000 user documents took 352.449µs
2016/04/06 08:27:50 PERFSTAT Range 0.0004 seconds
2016/04/06 08:27:50 Range Scan of 100000 user documents took 409.436µs
2016/04/06 08:27:52 PERFSTAT Range 1.9395 seconds
2016/04/06 08:27:52 Range Scan of 100000 user documents took 1.939597115s
2016/04/06 08:27:52 PERFSTAT Range 0.0004 seconds
2016/04/06 08:27:52 Range Scan of 100000 user documents took 466.146µs
2016/04/06 08:27:53 PERFSTAT Range 0.9073 seconds
2016/04/06 08:27:53 Range Scan of 100000 user documents took 907.370385ms
2016/04/06 08:27:53 PERFSTAT Range 0.1898 seconds
2016/04/06 08:27:53 Range Scan of 100000 user documents took 189.86332ms
2016/04/06 08:27:53 PERFSTAT Range 0.0004 seconds
2016/04/06 08:27:53 Range Scan of 100000 user documents took 470.81µs
2016/04/06 08:27:54 PERFSTAT Range 1.1019 seconds
2016/04/06 08:27:54 Range Scan of 100000 user documents took 1.10193078s
2016/04/06 08:27:55 PERFSTAT Range 1.2882 seconds
2016/04/06 08:27:55 Range Scan of 100000 user documents took 1.288277014s
2016/04/06 08:27:55 PERFSTAT Range 0.0005 seconds
2016/04/06 08:27:55 Range Scan of 100000 user documents took 525.428µs
2016/04/06 08:27:55 PERFSTAT Range 0.0004 seconds
2016/04/06 08:27:55 Range Scan of 100000 user documents took 395.232µs
2016/04/06 08:27:55 PERFSTAT Range 0.0004 seconds
2016/04/06 08:27:55 Range Scan of 100000 user documents took 452.692µs
2016/04/06 08:27:55 PERFSTAT Range 0.0004 seconds
2016/04/06 08:27:55 Range Scan of 100000 user documents took 403.579µs
2016/04/06 08:27:55 PERFSTAT Range 0.0004 seconds
2016/04/06 08:27:55 Range Scan of 100000 user documents took 404.424µs
2016/04/06 08:27:56 PERFSTAT Range 0.8041 seconds
2016/04/06 08:27:56 Range Scan of 100000 user documents took 804.113384ms
2016/04/06 08:27:56 PERFSTAT Range 0.0005 seconds
2016/04/06 08:27:56 Range Scan of 100000 user documents took 579.399µs
2016/04/06 08:27:57 PERFSTAT Range 1.1825 seconds
2016/04/06 08:27:57 Range Scan of 100000 user documents took 1.182587372s
2016/04/06 08:27:58 PERFSTAT Range 0.1449 seconds
2016/04/06 08:27:58 Range Scan of 100000 user documents took 144.977584ms
2016/04/06 08:27:58 PERFSTAT Range 0.0005 seconds
2016/04/06 08:27:58 Range Scan of 100000 user documents took 505.446µs
2016/04/06 08:27:58 PERFSTAT Range 0.0214 seconds
2016/04/06 08:27:58 Range Scan of 100000 user documents took 21.46443ms
2016/04/06 08:27:59 PERFSTAT Range 1.2698 seconds
2016/04/06 08:27:59 Range Scan of 100000 user documents took 1.269820098s
2016/04/06 08:27:59 PERFSTAT Range 0.0004 seconds
2016/04/06 08:27:59 Range Scan of 100000 user documents took 486.022µs
2016/04/06 08:27:59 PERFSTAT Range 0.0005 seconds
2016/04/06 08:27:59 Range Scan of 100000 user documents took 488.213µs
2016/04/06 08:27:59 PERFSTAT Range 0.0004 seconds
2016/04/06 08:27:59 Range Scan of 100000 user documents took 448.15µs
2016/04/06 08:27:59 Created the secondary index index_age. Waiting for it become active
2016/04/06 08:27:59 Index state is INDEX_STATE_ACTIVE
2016/04/06 08:27:59 Index is now active
2016/04/06 08:27:59 PERFSTAT CreateAndBuildIndex 28.1896 seconds
2016/04/06 08:27:59 PERFSTAT Range 0.5125 seconds
2016/04/06 08:27:59 Range Scan of 100000 user documents took 512.492679ms
2016/04/06 08:27:59 PERFSTAT Range 0.0004 seconds
2016/04/06 08:27:59 Range Scan of 100000 user documents took 469.771µs
2016/04/06 08:28:00 PERFSTAT Range 0.8217 seconds
2016/04/06 08:28:00 Range Scan of 100000 user documents took 821.756101ms
2016/04/06 08:28:00 PERFSTAT Range 0.0005 seconds
2016/04/06 08:28:00 Range Scan of 100000 user documents took 484.456µs
2016/04/06 08:28:01 PERFSTAT Range 0.6636 seconds
2016/04/06 08:28:01 Range Scan of 100000 user documents took 663.661824ms
2016/04/06 08:28:01 PERFSTAT Range 0.0026 seconds
2016/04/06 08:28:01 Range Scan of 100000 user documents took 2.674998ms
2016/04/06 08:28:01 PERFSTAT Range 0.0007 seconds
2016/04/06 08:28:01 Range Scan of 100000 user documents took 709.283µs
2016/04/06 08:28:01 PERFSTAT Range 0.4819 seconds
2016/04/06 08:28:01 Range Scan of 100000 user documents took 481.966461ms
2016/04/06 08:28:01 PERFSTAT Range 0.0005 seconds
2016/04/06 08:28:01 Range Scan of 100000 user documents took 903.569µs
2016/04/06 08:28:03 PERFSTAT Range 2.4883 seconds
2016/04/06 08:28:05 PERFSTAT Range 3.8867 seconds
2016/04/06 08:28:05 Range Scan of 100000 user documents took 3.886724836s
2016/04/06 08:28:05 PERFSTAT Range 0.0007 seconds
2016/04/06 08:28:05 Range Scan of 100000 user documents took 716.598µs
2016/04/06 08:28:07 PERFSTAT Range 1.8062 seconds
2016/04/06 08:28:07 Range Scan of 100000 user documents took 1.806276963s
2016/04/06 08:28:07 PERFSTAT Range 0.0005 seconds
2016/04/06 08:28:07 Range Scan of 100000 user documents took 546.248µs
2016/04/06 08:28:14 PERFSTAT Range 6.4380 seconds
2016/04/06 08:28:14 Range Scan of 100000 user documents took 6.438004881s
2016/04/06 08:28:15 PERFSTAT Range 1.6602 seconds
2016/04/06 08:28:15 Range Scan of 100000 user documents took 1.660338919s
2016/04/06 08:28:16 PERFSTAT Range 0.7850 seconds
2016/04/06 08:28:16 Range Scan of 100000 user documents took 785.044482ms
2016/04/06 08:28:17 PERFSTAT Range 1.0816 seconds
2016/04/06 08:28:17 Range Scan of 100000 user documents took 1.081662489s
2016/04/06 08:28:17 PERFSTAT Range 0.0005 seconds
2016/04/06 08:28:17 Range Scan of 100000 user documents took 536.935µs
2016/04/06 08:28:19 PERFSTAT Range 1.9362 seconds
2016/04/06 08:28:19 Range Scan of 100000 user documents took 1.936293312s
2016/04/06 08:28:19 PERFSTAT Range 0.0005 seconds
2016/04/06 08:28:19 Range Scan of 100000 user documents took 546.211µs
2016/04/06 08:28:19 PERFSTAT Range 0.0004 seconds
2016/04/06 08:28:19 Range Scan of 100000 user documents took 474.626µs
2016/04/06 08:28:22 PERFSTAT Range 2.5952 seconds
2016/04/06 08:28:22 Range Scan of 100000 user documents took 2.595283808s
2016/04/06 08:28:29 Created the secondary index index_firstname. Waiting for it become active
2016/04/06 08:28:29 Index state is INDEX_STATE_ACTIVE
2016/04/06 08:28:29 Index is now active
2016/04/06 08:28:29 PERFSTAT CreateAndBuildIndex 25.7602 seconds
2016/04/06 08:28:31 PERFSTAT Range 1.3416 seconds
2016/04/06 08:28:31 Dropping the secondary index index_age
2016/04/06 08:28:31 Index dropped
2016/04/06 08:28:31 PERFSTAT DropIndex 0.0411 seconds
2016/04/06 08:28:32 Dropping the secondary index index_firstname
2016/04/06 08:28:32 Index dropped
2016/04/06 08:28:32 PERFSTAT DropIndex 0.0251 seconds
--- PASS: TestScanWithConcurrentIndexOps (208.59s)
=== RUN   TestConcurrentScans_SameIndex
2016/04/06 08:28:33 In TestConcurrentScans_SameIndex()
2016/04/06 08:28:33 Generating JSON docs
2016/04/06 08:28:41 Setting initial JSON docs in KV
2016/04/06 08:29:57 All indexers are active
2016/04/06 08:29:57 Creating a 2i
2016/04/06 08:29:57 Index found:  index_company
2016/04/06 08:29:57 In Range Scan
2016/04/06 08:29:57 In Range Scan
2016/04/06 08:29:57 In Range Scan
2016/04/06 08:29:57 In Range Scan
2016/04/06 08:29:57 In Range Scan
2016/04/06 08:29:57 In Range Scan
2016/04/06 08:29:57 PERFSTAT Range 0.0020 seconds
2016/04/06 08:29:57 Range Scan of 100000 user documents took 2.105358ms
2016/04/06 08:29:57 PERFSTAT Range 0.0006 seconds
2016/04/06 08:29:57 Range Scan of 100000 user documents took 633.136µs
2016/04/06 08:29:57 PERFSTAT Range 0.0028 seconds
2016/04/06 08:29:57 Range Scan of 100000 user documents took 2.815757ms
2016/04/06 08:29:58 PERFSTAT Range 0.0043 seconds
2016/04/06 08:29:58 Range Scan of 100000 user documents took 4.309993ms
2016/04/06 08:29:58 PERFSTAT Range 0.0069 seconds
2016/04/06 08:29:58 Range Scan of 100000 user documents took 6.925205ms
2016/04/06 08:29:58 PERFSTAT Range 0.0005 seconds
2016/04/06 08:29:58 Range Scan of 100000 user documents took 508.94µs
2016/04/06 08:30:06 PERFSTAT Range 8.2153 seconds
2016/04/06 08:30:06 Range Scan of 100000 user documents took 8.220496085s
2016/04/06 08:30:08 PERFSTAT Range 10.2341 seconds
2016/04/06 08:30:08 Range Scan of 100000 user documents took 10.234632263s
2016/04/06 08:30:09 PERFSTAT Range 11.0707 seconds
2016/04/06 08:30:09 Range Scan of 100000 user documents took 11.070784411s
2016/04/06 08:30:09 PERFSTAT Range 0.0008 seconds
2016/04/06 08:30:09 Range Scan of 100000 user documents took 847.604µs
2016/04/06 08:30:09 PERFSTAT Range 11.2760 seconds
2016/04/06 08:30:09 Range Scan of 100000 user documents took 11.275996318s
2016/04/06 08:30:10 PERFSTAT Range 12.5408 seconds
2016/04/06 08:30:10 Range Scan of 100000 user documents took 12.540904318s
2016/04/06 08:30:10 PERFSTAT Range 12.6876 seconds
2016/04/06 08:30:10 Range Scan of 100000 user documents took 12.68769098s
2016/04/06 08:30:10 PERFSTAT Range 1.9290 seconds
2016/04/06 08:30:10 Range Scan of 100000 user documents took 1.929089301s
2016/04/06 08:30:13 PERFSTAT Range 5.3318 seconds
2016/04/06 08:30:13 Range Scan of 100000 user documents took 5.332108575s
2016/04/06 08:30:14 PERFSTAT Range 3.6221 seconds
2016/04/06 08:30:14 Range Scan of 100000 user documents took 3.622272475s
2016/04/06 08:30:14 PERFSTAT Range 0.0800 seconds
2016/04/06 08:30:14 Range Scan of 100000 user documents took 80.075792ms
2016/04/06 08:30:14 PERFSTAT Range 3.2598 seconds
2016/04/06 08:30:14 Range Scan of 100000 user documents took 3.259860154s
2016/04/06 08:30:14 PERFSTAT Range 0.8487 seconds
2016/04/06 08:30:14 Range Scan of 100000 user documents took 848.882455ms
2016/04/06 08:30:14 PERFSTAT Range 0.0120 seconds
2016/04/06 08:30:14 Range Scan of 100000 user documents took 12.057012ms
2016/04/06 08:30:14 PERFSTAT Range 0.0108 seconds
2016/04/06 08:30:14 Range Scan of 100000 user documents took 10.938081ms
2016/04/06 08:30:14 PERFSTAT Range 0.0147 seconds
2016/04/06 08:30:14 Range Scan of 100000 user documents took 14.765552ms
2016/04/06 08:30:14 PERFSTAT Range 0.4948 seconds
2016/04/06 08:30:14 Range Scan of 100000 user documents took 494.864443ms
2016/04/06 08:30:14 PERFSTAT Range 0.0998 seconds
2016/04/06 08:30:14 Range Scan of 100000 user documents took 99.87906ms
2016/04/06 08:30:16 PERFSTAT Range 9.8417 seconds
2016/04/06 08:30:16 Range Scan of 100000 user documents took 9.841808908s
2016/04/06 08:30:16 PERFSTAT Range 6.9878 seconds
2016/04/06 08:30:16 Range Scan of 100000 user documents took 6.987875276s
2016/04/06 08:30:16 PERFSTAT Range 6.1972 seconds
2016/04/06 08:30:16 Range Scan of 100000 user documents took 6.197331929s
2016/04/06 08:30:16 PERFSTAT Range 0.0017 seconds
2016/04/06 08:30:16 Range Scan of 100000 user documents took 1.770677ms
2016/04/06 08:30:16 PERFSTAT Range 0.0023 seconds
2016/04/06 08:30:16 Range Scan of 100000 user documents took 2.301796ms
2016/04/06 08:30:17 PERFSTAT Range 2.7482 seconds
2016/04/06 08:30:17 Range Scan of 100000 user documents took 2.748257084s
2016/04/06 08:30:18 PERFSTAT Range 0.7707 seconds
2016/04/06 08:30:18 Range Scan of 100000 user documents took 770.738091ms
2016/04/06 08:30:18 PERFSTAT Range 0.0052 seconds
2016/04/06 08:30:18 Range Scan of 100000 user documents took 5.263186ms
2016/04/06 08:30:18 PERFSTAT Range 2.1507 seconds
2016/04/06 08:30:18 Range Scan of 100000 user documents took 2.150781381s
2016/04/06 08:30:20 PERFSTAT Range 6.1581 seconds
2016/04/06 08:30:20 Range Scan of 100000 user documents took 6.158292115s
2016/04/06 08:30:20 PERFSTAT Range 0.0007 seconds
2016/04/06 08:30:20 Range Scan of 100000 user documents took 693.793µs
2016/04/06 08:30:20 PERFSTAT Range 0.0004 seconds
2016/04/06 08:30:20 Range Scan of 100000 user documents took 403.188µs
2016/04/06 08:30:20 PERFSTAT Range 6.4193 seconds
2016/04/06 08:30:20 Range Scan of 100000 user documents took 6.419347264s
2016/04/06 08:30:20 PERFSTAT Range 0.0126 seconds
2016/04/06 08:30:20 Range Scan of 100000 user documents took 12.657369ms
2016/04/06 08:30:20 PERFSTAT Range 2.2456 seconds
2016/04/06 08:30:20 Range Scan of 100000 user documents took 2.245656606s
2016/04/06 08:30:20 PERFSTAT Range 0.0443 seconds
2016/04/06 08:30:20 Range Scan of 100000 user documents took 44.385194ms
2016/04/06 08:30:22 PERFSTAT Range 6.6465 seconds
2016/04/06 08:30:22 Range Scan of 100000 user documents took 6.64665484s
2016/04/06 08:30:22 PERFSTAT Range 0.0014 seconds
2016/04/06 08:30:22 Range Scan of 100000 user documents took 1.501399ms
2016/04/06 08:30:22 PERFSTAT Range 2.0901 seconds
2016/04/06 08:30:22 Range Scan of 100000 user documents took 2.090166879s
2016/04/06 08:30:23 PERFSTAT Range 0.6379 seconds
2016/04/06 08:30:23 Range Scan of 100000 user documents took 637.9907ms
2016/04/06 08:30:23 PERFSTAT Range 0.0169 seconds
2016/04/06 08:30:23 Range Scan of 100000 user documents took 16.975722ms
2016/04/06 08:30:23 PERFSTAT Range 0.0144 seconds
2016/04/06 08:30:23 Range Scan of 100000 user documents took 14.416771ms
2016/04/06 08:30:23 PERFSTAT Range 0.0025 seconds
2016/04/06 08:30:23 Range Scan of 100000 user documents took 7.275656ms
2016/04/06 08:30:24 PERFSTAT Range 1.2458 seconds
2016/04/06 08:30:24 Range Scan of 100000 user documents took 1.245904276s
2016/04/06 08:30:24 PERFSTAT Range 0.0213 seconds
2016/04/06 08:30:24 Range Scan of 100000 user documents took 21.386116ms
2016/04/06 08:30:24 PERFSTAT Range 0.0098 seconds
2016/04/06 08:30:24 Range Scan of 100000 user documents took 9.858465ms
2016/04/06 08:30:24 PERFSTAT Range 0.0112 seconds
2016/04/06 08:30:24 Range Scan of 100000 user documents took 11.210537ms
2016/04/06 08:30:26 PERFSTAT Range 9.4505 seconds
2016/04/06 08:30:26 Range Scan of 100000 user documents took 9.450610565s
2016/04/06 08:30:26 PERFSTAT Range 0.0078 seconds
2016/04/06 08:30:26 Range Scan of 100000 user documents took 7.816484ms
2016/04/06 08:30:26 PERFSTAT Range 0.0785 seconds
2016/04/06 08:30:26 Range Scan of 100000 user documents took 78.561648ms
2016/04/06 08:30:26 PERFSTAT Range 6.3611 seconds
2016/04/06 08:30:26 Range Scan of 100000 user documents took 6.361187096s
2016/04/06 08:30:28 PERFSTAT Range 10.3817 seconds
2016/04/06 08:30:28 Range Scan of 100000 user documents took 10.381767286s
2016/04/06 08:30:29 PERFSTAT Range 3.3610 seconds
2016/04/06 08:30:29 Range Scan of 100000 user documents took 3.361124334s
2016/04/06 08:30:30 PERFSTAT Range 7.5167 seconds
2016/04/06 08:30:30 Range Scan of 100000 user documents took 7.516829327s
2016/04/06 08:30:31 PERFSTAT Range 6.3544 seconds
2016/04/06 08:30:31 Range Scan of 100000 user documents took 6.357305422s
2016/04/06 08:30:33 PERFSTAT Range 13.0024 seconds
2016/04/06 08:30:33 Range Scan of 100000 user documents took 13.002509029s
2016/04/06 08:30:33 PERFSTAT Range 0.0171 seconds
2016/04/06 08:30:33 Range Scan of 100000 user documents took 17.123311ms
2016/04/06 08:30:33 PERFSTAT Range 0.0264 seconds
2016/04/06 08:30:33 Range Scan of 100000 user documents took 26.427181ms
2016/04/06 08:30:33 PERFSTAT Range 5.0217 seconds
2016/04/06 08:30:33 Range Scan of 100000 user documents took 5.021750449s
2016/04/06 08:30:33 PERFSTAT Range 0.0159 seconds
2016/04/06 08:30:33 Range Scan of 100000 user documents took 15.984979ms
2016/04/06 08:30:33 PERFSTAT Range 0.0360 seconds
2016/04/06 08:30:33 Range Scan of 100000 user documents took 36.051138ms
2016/04/06 08:30:33 PERFSTAT Range 0.0109 seconds
2016/04/06 08:30:33 Range Scan of 100000 user documents took 10.973889ms
2016/04/06 08:30:33 PERFSTAT Range 2.7068 seconds
2016/04/06 08:30:33 Range Scan of 100000 user documents took 2.706875916s
2016/04/06 08:30:34 PERFSTAT Range 4.3212 seconds
2016/04/06 08:30:34 Range Scan of 100000 user documents took 4.321293762s
2016/04/06 08:30:35 PERFSTAT Range 6.1024 seconds
2016/04/06 08:30:35 Range Scan of 100000 user documents took 6.10257047s
2016/04/06 08:30:35 PERFSTAT Range 0.0084 seconds
2016/04/06 08:30:35 Range Scan of 100000 user documents took 18.391413ms
2016/04/06 08:30:36 PERFSTAT Range 2.6376 seconds
2016/04/06 08:30:36 Range Scan of 100000 user documents took 2.637705961s
2016/04/06 08:30:36 PERFSTAT Range 0.0142 seconds
2016/04/06 08:30:36 Range Scan of 100000 user documents took 14.20013ms
2016/04/06 08:30:36 PERFSTAT Range 0.0115 seconds
2016/04/06 08:30:36 Range Scan of 100000 user documents took 11.603497ms
2016/04/06 08:30:37 PERFSTAT Range 3.0830 seconds
2016/04/06 08:30:37 Range Scan of 100000 user documents took 3.091102266s
2016/04/06 08:30:37 PERFSTAT Range 0.0016 seconds
2016/04/06 08:30:37 Range Scan of 100000 user documents took 1.680351ms
2016/04/06 08:30:37 PERFSTAT Range 0.0008 seconds
2016/04/06 08:30:37 Range Scan of 100000 user documents took 832.424µs
2016/04/06 08:30:37 PERFSTAT Range 0.0006 seconds
2016/04/06 08:30:37 Range Scan of 100000 user documents took 621.772µs
2016/04/06 08:30:37 PERFSTAT Range 0.0130 seconds
2016/04/06 08:30:37 Range Scan of 100000 user documents took 13.000035ms
2016/04/06 08:30:37 PERFSTAT Range 0.0190 seconds
2016/04/06 08:30:37 Range Scan of 100000 user documents took 19.073109ms
2016/04/06 08:30:37 PERFSTAT Range 0.0096 seconds
2016/04/06 08:30:37 Range Scan of 100000 user documents took 9.675048ms
2016/04/06 08:30:37 PERFSTAT Range 0.0007 seconds
2016/04/06 08:30:37 Range Scan of 100000 user documents took 707.385µs
2016/04/06 08:30:38 PERFSTAT Range 11.4345 seconds
2016/04/06 08:30:38 Range Scan of 100000 user documents took 11.434657515s
2016/04/06 08:30:38 PERFSTAT Range 0.0243 seconds
2016/04/06 08:30:38 Range Scan of 100000 user documents took 24.32237ms
2016/04/06 08:30:38 PERFSTAT Range 0.0239 seconds
2016/04/06 08:30:38 Range Scan of 100000 user documents took 23.990425ms
2016/04/06 08:30:38 PERFSTAT Range 0.0074 seconds
2016/04/06 08:30:38 Range Scan of 100000 user documents took 7.476803ms
2016/04/06 08:30:38 PERFSTAT Range 0.0166 seconds
2016/04/06 08:30:38 Range Scan of 100000 user documents took 16.584287ms
2016/04/06 08:30:39 PERFSTAT Range 0.6681 seconds
2016/04/06 08:30:39 Range Scan of 100000 user documents took 668.363925ms
2016/04/06 08:30:39 PERFSTAT Range 1.5650 seconds
2016/04/06 08:30:39 Range Scan of 100000 user documents took 1.565129589s
2016/04/06 08:30:39 PERFSTAT Range 0.0139 seconds
2016/04/06 08:30:39 Range Scan of 100000 user documents took 13.903308ms
2016/04/06 08:30:39 PERFSTAT Range 5.9437 seconds
2016/04/06 08:30:39 Range Scan of 100000 user documents took 5.943856156s
2016/04/06 08:30:39 PERFSTAT Range 3.7519 seconds
2016/04/06 08:30:39 Range Scan of 100000 user documents took 3.751973526s
2016/04/06 08:30:39 PERFSTAT Range 0.0583 seconds
2016/04/06 08:30:39 Range Scan of 100000 user documents took 58.336396ms
2016/04/06 08:30:40 PERFSTAT Range 1.6841 seconds
2016/04/06 08:30:40 Range Scan of 100000 user documents took 1.684185508s
2016/04/06 08:30:40 PERFSTAT Range 0.0006 seconds
2016/04/06 08:30:40 Range Scan of 100000 user documents took 589.911µs
2016/04/06 08:30:40 PERFSTAT Range 0.0549 seconds
2016/04/06 08:30:40 Range Scan of 100000 user documents took 55.051621ms
2016/04/06 08:30:42 PERFSTAT Range 1.5488 seconds
2016/04/06 08:30:42 Range Scan of 100000 user documents took 1.548908488s
2016/04/06 08:30:42 PERFSTAT Range 0.0066 seconds
2016/04/06 08:30:42 Range Scan of 100000 user documents took 7.309429ms
2016/04/06 08:30:42 PERFSTAT Range 0.0050 seconds
2016/04/06 08:30:42 Range Scan of 100000 user documents took 8.006018ms
2016/04/06 08:30:42 PERFSTAT Range 6.2669 seconds
2016/04/06 08:30:42 Range Scan of 100000 user documents took 6.267017652s
2016/04/06 08:30:42 PERFSTAT Range 0.0011 seconds
2016/04/06 08:30:42 Range Scan of 100000 user documents took 1.134745ms
2016/04/06 08:30:43 PERFSTAT Range 3.4212 seconds
2016/04/06 08:30:43 Range Scan of 100000 user documents took 3.449701162s
2016/04/06 08:30:43 PERFSTAT Range 0.0111 seconds
2016/04/06 08:30:43 Range Scan of 100000 user documents took 12.091656ms
2016/04/06 08:30:43 PERFSTAT Range 0.0038 seconds
2016/04/06 08:30:43 Range Scan of 100000 user documents took 3.88173ms
2016/04/06 08:30:43 PERFSTAT Range 0.0083 seconds
2016/04/06 08:30:43 Range Scan of 100000 user documents took 8.341336ms
2016/04/06 08:30:44 PERFSTAT Range 5.4485 seconds
2016/04/06 08:30:44 Range Scan of 100000 user documents took 5.448556376s
2016/04/06 08:30:44 PERFSTAT Range 0.0175 seconds
2016/04/06 08:30:44 Range Scan of 100000 user documents took 17.498112ms
2016/04/06 08:30:44 PERFSTAT Range 0.0644 seconds
2016/04/06 08:30:44 Range Scan of 100000 user documents took 64.499756ms
2016/04/06 08:30:44 PERFSTAT Range 0.0041 seconds
2016/04/06 08:30:44 Range Scan of 100000 user documents took 4.187868ms
2016/04/06 08:30:44 PERFSTAT Range 0.0077 seconds
2016/04/06 08:30:44 Range Scan of 100000 user documents took 7.724773ms
2016/04/06 08:30:45 PERFSTAT Range 2.6045 seconds
2016/04/06 08:30:45 Range Scan of 100000 user documents took 2.604564215s
2016/04/06 08:30:45 PERFSTAT Range 11.5786 seconds
2016/04/06 08:30:45 Range Scan of 100000 user documents took 11.578691165s
2016/04/06 08:30:45 PERFSTAT Range 0.0162 seconds
2016/04/06 08:30:45 Range Scan of 100000 user documents took 16.217229ms
2016/04/06 08:30:45 PERFSTAT Range 0.4285 seconds
2016/04/06 08:30:45 Range Scan of 100000 user documents took 428.647844ms
2016/04/06 08:30:48 PERFSTAT Range 2.7951 seconds
2016/04/06 08:30:48 Range Scan of 100000 user documents took 2.795196334s
2016/04/06 08:30:48 PERFSTAT Range 8.9323 seconds
2016/04/06 08:30:48 Range Scan of 100000 user documents took 8.932405579s
2016/04/06 08:30:48 PERFSTAT Range 0.0004 seconds
2016/04/06 08:30:48 Range Scan of 100000 user documents took 468.336µs
2016/04/06 08:30:49 PERFSTAT Range 6.7692 seconds
2016/04/06 08:30:49 Range Scan of 100000 user documents took 6.769333985s
2016/04/06 08:30:49 PERFSTAT Range 0.0221 seconds
2016/04/06 08:30:49 Range Scan of 100000 user documents took 22.110942ms
2016/04/06 08:30:49 PERFSTAT Range 0.0141 seconds
2016/04/06 08:30:49 Range Scan of 100000 user documents took 14.106808ms
2016/04/06 08:30:49 PERFSTAT Range 0.0193 seconds
2016/04/06 08:30:49 Range Scan of 100000 user documents took 19.392933ms
2016/04/06 08:30:49 PERFSTAT Range 0.0224 seconds
2016/04/06 08:30:49 Range Scan of 100000 user documents took 22.403785ms
2016/04/06 08:30:49 PERFSTAT Range 4.1890 seconds
2016/04/06 08:30:49 Range Scan of 100000 user documents took 4.189114106s
2016/04/06 08:30:51 PERFSTAT Range 6.8154 seconds
2016/04/06 08:30:51 Range Scan of 100000 user documents took 6.816764367s
2016/04/06 08:30:51 PERFSTAT Range 8.6608 seconds
2016/04/06 08:30:51 Range Scan of 100000 user documents took 8.660835371s
2016/04/06 08:30:52 PERFSTAT Range 4.2812 seconds
2016/04/06 08:30:52 Range Scan of 100000 user documents took 4.281212745s
2016/04/06 08:30:53 PERFSTAT Range 4.3830 seconds
2016/04/06 08:30:53 Range Scan of 100000 user documents took 4.38307896s
2016/04/06 08:30:53 PERFSTAT Range 0.0167 seconds
2016/04/06 08:30:53 Range Scan of 100000 user documents took 16.776479ms
2016/04/06 08:30:54 PERFSTAT Range 0.0613 seconds
2016/04/06 08:30:54 Range Scan of 100000 user documents took 61.443234ms
2016/04/06 08:30:54 PERFSTAT Range 0.0189 seconds
2016/04/06 08:30:54 Range Scan of 100000 user documents took 18.93776ms
2016/04/06 08:30:54 PERFSTAT Range 2.3402 seconds
2016/04/06 08:30:54 Range Scan of 100000 user documents took 2.34032524s
2016/04/06 08:30:54 PERFSTAT Range 4.7236 seconds
2016/04/06 08:30:54 Range Scan of 100000 user documents took 4.723742619s
2016/04/06 08:30:54 PERFSTAT Range 0.0503 seconds
2016/04/06 08:30:54 Range Scan of 100000 user documents took 50.360349ms
2016/04/06 08:30:54 PERFSTAT Range 0.0121 seconds
2016/04/06 08:30:54 Range Scan of 100000 user documents took 12.1047ms
2016/04/06 08:30:54 PERFSTAT Range 0.4248 seconds
2016/04/06 08:30:54 Range Scan of 100000 user documents took 424.887381ms
2016/04/06 08:30:55 PERFSTAT Range 6.7361 seconds
2016/04/06 08:30:55 Range Scan of 100000 user documents took 6.736188222s
2016/04/06 08:30:55 PERFSTAT Range 0.0102 seconds
2016/04/06 08:30:55 Range Scan of 100000 user documents took 10.292906ms
2016/04/06 08:30:55 PERFSTAT Range 0.9025 seconds
2016/04/06 08:30:55 Range Scan of 100000 user documents took 902.608773ms
2016/04/06 08:30:55 PERFSTAT Range 1.2528 seconds
2016/04/06 08:30:55 Range Scan of 100000 user documents took 1.253566861s
2016/04/06 08:30:56 PERFSTAT Range 4.4742 seconds
2016/04/06 08:30:56 Range Scan of 100000 user documents took 4.474336853s
2016/04/06 08:30:58 PERFSTAT Range 1.0919 seconds
2016/04/06 08:30:58 Range Scan of 100000 user documents took 1.091963364s
2016/04/06 08:30:58 PERFSTAT Range 3.8702 seconds
2016/04/06 08:30:58 Range Scan of 100000 user documents took 3.870290641s
2016/04/06 08:30:58 PERFSTAT Range 0.0484 seconds
2016/04/06 08:30:58 Range Scan of 100000 user documents took 48.435094ms
2016/04/06 08:30:58 PERFSTAT Range 0.0018 seconds
2016/04/06 08:30:58 Range Scan of 100000 user documents took 1.822021ms
2016/04/06 08:30:58 PERFSTAT Range 0.0089 seconds
2016/04/06 08:30:58 Range Scan of 100000 user documents took 8.932226ms
2016/04/06 08:30:58 PERFSTAT Range 0.0112 seconds
2016/04/06 08:30:58 Range Scan of 100000 user documents took 11.242619ms
2016/04/06 08:30:58 PERFSTAT Range 0.0237 seconds
2016/04/06 08:30:58 Range Scan of 100000 user documents took 23.767605ms
2016/04/06 08:30:59 PERFSTAT Range 7.5809 seconds
2016/04/06 08:30:59 Range Scan of 100000 user documents took 7.580965584s
2016/04/06 08:31:00 PERFSTAT Range 1.8193 seconds
2016/04/06 08:31:00 Range Scan of 100000 user documents took 1.819395169s
2016/04/06 08:31:01 PERFSTAT Range 5.6414 seconds
2016/04/06 08:31:01 Range Scan of 100000 user documents took 5.6414175s
2016/04/06 08:31:01 PERFSTAT Range 0.0004 seconds
2016/04/06 08:31:01 Range Scan of 100000 user documents took 467.276µs
2016/04/06 08:31:01 PERFSTAT Range 0.1933 seconds
2016/04/06 08:31:01 Range Scan of 100000 user documents took 193.416154ms
2016/04/06 08:31:01 PERFSTAT Range 0.0014 seconds
2016/04/06 08:31:01 Range Scan of 100000 user documents took 1.410969ms
2016/04/06 08:31:01 PERFSTAT Range 0.0004 seconds
2016/04/06 08:31:01 Range Scan of 100000 user documents took 425.213µs
2016/04/06 08:31:01 PERFSTAT Range 0.0003 seconds
2016/04/06 08:31:01 Range Scan of 100000 user documents took 369.359µs
2016/04/06 08:31:01 PERFSTAT Range 6.0150 seconds
2016/04/06 08:31:01 Range Scan of 100000 user documents took 6.015062703s
2016/04/06 08:31:01 PERFSTAT Range 0.0168 seconds
2016/04/06 08:31:01 Range Scan of 100000 user documents took 16.882895ms
2016/04/06 08:31:02 PERFSTAT Range 0.7460 seconds
2016/04/06 08:31:02 Range Scan of 100000 user documents took 746.135828ms
2016/04/06 08:31:02 PERFSTAT Range 0.0332 seconds
2016/04/06 08:31:02 Range Scan of 100000 user documents took 33.295254ms
2016/04/06 08:31:02 PERFSTAT Range 6.8267 seconds
2016/04/06 08:31:02 Range Scan of 100000 user documents took 6.826854856s
2016/04/06 08:31:02 PERFSTAT Range 3.0942 seconds
2016/04/06 08:31:02 Range Scan of 100000 user documents took 3.094246827s
2016/04/06 08:31:03 PERFSTAT Range 2.0056 seconds
2016/04/06 08:31:03 Range Scan of 100000 user documents took 2.00587261s
2016/04/06 08:31:03 PERFSTAT Range 0.0173 seconds
2016/04/06 08:31:03 Range Scan of 100000 user documents took 17.397764ms
2016/04/06 08:31:03 PERFSTAT Range 0.0004 seconds
2016/04/06 08:31:03 Range Scan of 100000 user documents took 419.792µs
2016/04/06 08:31:03 PERFSTAT Range 0.0004 seconds
2016/04/06 08:31:03 Range Scan of 100000 user documents took 409.18µs
2016/04/06 08:31:03 PERFSTAT Range 0.0004 seconds
2016/04/06 08:31:03 Range Scan of 100000 user documents took 381.654µs
2016/04/06 08:31:03 PERFSTAT Range 0.0003 seconds
2016/04/06 08:31:03 Range Scan of 100000 user documents took 345.384µs
2016/04/06 08:31:03 PERFSTAT Range 5.3982 seconds
2016/04/06 08:31:03 Range Scan of 100000 user documents took 5.398698128s
2016/04/06 08:31:04 PERFSTAT Range 2.0243 seconds
2016/04/06 08:31:04 Range Scan of 100000 user documents took 2.033593817s
2016/04/06 08:31:04 PERFSTAT Range 0.0016 seconds
2016/04/06 08:31:04 Range Scan of 100000 user documents took 1.675677ms
2016/04/06 08:31:04 PERFSTAT Range 1.5226 seconds
2016/04/06 08:31:04 Range Scan of 100000 user documents took 1.522758339s
2016/04/06 08:31:05 PERFSTAT Range 1.7071 seconds
2016/04/06 08:31:05 Range Scan of 100000 user documents took 1.707154206s
2016/04/06 08:31:07 PERFSTAT Range 1.9373 seconds
2016/04/06 08:31:07 Range Scan of 100000 user documents took 1.937350964s
2016/04/06 08:31:07 PERFSTAT Range 5.0576 seconds
2016/04/06 08:31:07 Range Scan of 100000 user documents took 5.05768588s
2016/04/06 08:31:07 PERFSTAT Range 7.4790 seconds
2016/04/06 08:31:07 Range Scan of 100000 user documents took 7.480380416s
2016/04/06 08:31:07 PERFSTAT Range 0.0458 seconds
2016/04/06 08:31:07 Range Scan of 100000 user documents took 45.806423ms
2016/04/06 08:31:08 PERFSTAT Range 1.6299 seconds
2016/04/06 08:31:08 Range Scan of 100000 user documents took 1.629928598s
2016/04/06 08:31:08 PERFSTAT Range 0.0022 seconds
2016/04/06 08:31:08 Range Scan of 100000 user documents took 2.259593ms
2016/04/06 08:31:08 PERFSTAT Range 0.0154 seconds
2016/04/06 08:31:08 Range Scan of 100000 user documents took 15.466441ms
2016/04/06 08:31:08 PERFSTAT Range 0.0139 seconds
2016/04/06 08:31:08 Range Scan of 100000 user documents took 13.960954ms
2016/04/06 08:31:09 PERFSTAT Range 7.2142 seconds
2016/04/06 08:31:09 Range Scan of 100000 user documents took 7.214334175s
2016/04/06 08:31:13 PERFSTAT Range 4.9622 seconds
2016/04/06 08:31:13 Range Scan of 100000 user documents took 4.962317127s
2016/04/06 08:31:13 PERFSTAT Range 0.0038 seconds
2016/04/06 08:31:13 Range Scan of 100000 user documents took 3.864691ms
2016/04/06 08:31:13 PERFSTAT Range 8.9214 seconds
2016/04/06 08:31:13 Range Scan of 100000 user documents took 8.921473173s
2016/04/06 08:31:13 PERFSTAT Range 0.0199 seconds
2016/04/06 08:31:13 Range Scan of 100000 user documents took 20.028126ms
2016/04/06 08:31:14 PERFSTAT Range 10.1994 seconds
2016/04/06 08:31:14 Range Scan of 100000 user documents took 10.199456079s
2016/04/06 08:31:14 PERFSTAT Range 0.0293 seconds
2016/04/06 08:31:14 Range Scan of 100000 user documents took 29.329295ms
2016/04/06 08:31:14 PERFSTAT Range 0.0549 seconds
2016/04/06 08:31:14 Range Scan of 100000 user documents took 54.932468ms
2016/04/06 08:31:14 PERFSTAT Range 7.7089 seconds
2016/04/06 08:31:14 Range Scan of 100000 user documents took 7.709013033s
2016/04/06 08:31:14 PERFSTAT Range 0.0057 seconds
2016/04/06 08:31:14 Range Scan of 100000 user documents took 5.755ms
2016/04/06 08:31:16 PERFSTAT Range 7.0344 seconds
2016/04/06 08:31:16 Range Scan of 100000 user documents took 7.034434636s
2016/04/06 08:31:17 PERFSTAT Range 2.7226 seconds
2016/04/06 08:31:17 Range Scan of 100000 user documents took 2.729791117s
2016/04/06 08:31:19 PERFSTAT Range 11.3392 seconds
2016/04/06 08:31:19 Range Scan of 100000 user documents took 11.339290322s
2016/04/06 08:31:19 PERFSTAT Range 5.6573 seconds
2016/04/06 08:31:19 Range Scan of 100000 user documents took 5.657457748s
2016/04/06 08:31:19 PERFSTAT Range 0.1149 seconds
2016/04/06 08:31:19 Range Scan of 100000 user documents took 114.962385ms
2016/04/06 08:31:19 PERFSTAT Range 0.0078 seconds
2016/04/06 08:31:19 Range Scan of 100000 user documents took 7.822877ms
2016/04/06 08:31:21 PERFSTAT Range 4.6839 seconds
2016/04/06 08:31:21 Range Scan of 100000 user documents took 4.684048552s
2016/04/06 08:31:21 PERFSTAT Range 0.0497 seconds
2016/04/06 08:31:21 Range Scan of 100000 user documents took 49.842374ms
2016/04/06 08:31:21 PERFSTAT Range 0.0453 seconds
2016/04/06 08:31:21 Range Scan of 100000 user documents took 45.376648ms
2016/04/06 08:31:23 PERFSTAT Range 10.2816 seconds
2016/04/06 08:31:23 Range Scan of 100000 user documents took 10.281670008s
2016/04/06 08:31:24 PERFSTAT Range 5.4242 seconds
2016/04/06 08:31:24 Range Scan of 100000 user documents took 5.424328357s
2016/04/06 08:31:25 PERFSTAT Range 10.3609 seconds
2016/04/06 08:31:25 Range Scan of 100000 user documents took 10.360954228s
2016/04/06 08:31:25 PERFSTAT Range 0.0190 seconds
2016/04/06 08:31:25 Range Scan of 100000 user documents took 18.99671ms
2016/04/06 08:31:25 PERFSTAT Range 0.0065 seconds
2016/04/06 08:31:25 Range Scan of 100000 user documents took 6.552598ms
2016/04/06 08:31:25 PERFSTAT Range 0.0093 seconds
2016/04/06 08:31:25 Range Scan of 100000 user documents took 9.35319ms
2016/04/06 08:31:25 PERFSTAT Range 0.0124 seconds
2016/04/06 08:31:25 Range Scan of 100000 user documents took 12.435994ms
2016/04/06 08:31:25 PERFSTAT Range 0.0213 seconds
2016/04/06 08:31:25 Range Scan of 100000 user documents took 21.361309ms
2016/04/06 08:31:27 PERFSTAT Range 2.1285 seconds
2016/04/06 08:31:27 Range Scan of 100000 user documents took 2.128622272s
2016/04/06 08:31:30 PERFSTAT Range 10.9778 seconds
2016/04/06 08:31:30 Range Scan of 100000 user documents took 10.977979262s
2016/04/06 08:31:30 PERFSTAT Range 12.8073 seconds
2016/04/06 08:31:30 Range Scan of 100000 user documents took 12.807335374s
2016/04/06 08:31:30 PERFSTAT Range 0.0763 seconds
2016/04/06 08:31:30 Range Scan of 100000 user documents took 76.325813ms
2016/04/06 08:31:31 PERFSTAT Range 4.3150 seconds
2016/04/06 08:31:31 Range Scan of 100000 user documents took 4.318066233s
2016/04/06 08:31:31 PERFSTAT Range 0.0107 seconds
2016/04/06 08:31:31 Range Scan of 100000 user documents took 10.70136ms
2016/04/06 08:31:34 PERFSTAT Range 8.7202 seconds
2016/04/06 08:31:34 Range Scan of 100000 user documents took 8.726884944s
2016/04/06 08:31:34 PERFSTAT Range 0.0143 seconds
2016/04/06 08:31:34 Range Scan of 100000 user documents took 14.422061ms
2016/04/06 08:31:34 PERFSTAT Range 0.0153 seconds
2016/04/06 08:31:34 Range Scan of 100000 user documents took 15.320983ms
2016/04/06 08:31:34 PERFSTAT Range 0.0018 seconds
2016/04/06 08:31:34 Range Scan of 100000 user documents took 1.813218ms
2016/04/06 08:31:34 PERFSTAT Range 0.0010 seconds
2016/04/06 08:31:34 Range Scan of 100000 user documents took 1.07204ms
2016/04/06 08:31:34 PERFSTAT Range 10.2967 seconds
2016/04/06 08:31:34 Range Scan of 100000 user documents took 10.296731659s
2016/04/06 08:31:34 PERFSTAT Range 0.0163 seconds
2016/04/06 08:31:34 Range Scan of 100000 user documents took 16.365576ms
2016/04/06 08:31:34 PERFSTAT Range 0.6084 seconds
2016/04/06 08:31:34 Range Scan of 100000 user documents took 608.450613ms
2016/04/06 08:31:34 PERFSTAT Range 0.0080 seconds
2016/04/06 08:31:34 Range Scan of 100000 user documents took 8.056601ms
2016/04/06 08:31:34 PERFSTAT Range 0.0196 seconds
2016/04/06 08:31:34 Range Scan of 100000 user documents took 19.628831ms
2016/04/06 08:31:34 PERFSTAT Range 13.7083 seconds
2016/04/06 08:31:34 Range Scan of 100000 user documents took 13.708369312s
2016/04/06 08:31:35 PERFSTAT Range 1.3167 seconds
2016/04/06 08:31:35 Range Scan of 100000 user documents took 1.316776109s
2016/04/06 08:31:35 PERFSTAT Range 0.0078 seconds
2016/04/06 08:31:35 Range Scan of 100000 user documents took 7.818944ms
2016/04/06 08:31:36 PERFSTAT Range 6.8531 seconds
2016/04/06 08:31:36 Range Scan of 100000 user documents took 6.853163665s
2016/04/06 08:31:38 PERFSTAT Range 1.5911 seconds
2016/04/06 08:31:38 Range Scan of 100000 user documents took 1.591162735s
2016/04/06 08:31:40 PERFSTAT Range 9.3155 seconds
2016/04/06 08:31:40 Range Scan of 100000 user documents took 9.31984182s
2016/04/06 08:31:40 PERFSTAT Range 0.1783 seconds
2016/04/06 08:31:40 Range Scan of 100000 user documents took 178.467973ms
2016/04/06 08:31:41 PERFSTAT Range 0.1759 seconds
2016/04/06 08:31:41 Range Scan of 100000 user documents took 175.943877ms
2016/04/06 08:31:41 PERFSTAT Range 6.8870 seconds
2016/04/06 08:31:41 Range Scan of 100000 user documents took 6.889629524s
2016/04/06 08:31:41 PERFSTAT Range 11.5911 seconds
2016/04/06 08:31:41 Range Scan of 100000 user documents took 11.591196603s
2016/04/06 08:31:41 PERFSTAT Range 0.0043 seconds
2016/04/06 08:31:41 Range Scan of 100000 user documents took 4.330053ms
2016/04/06 08:31:42 PERFSTAT Range 7.3587 seconds
2016/04/06 08:31:42 Range Scan of 100000 user documents took 7.358783892s
2016/04/06 08:31:42 PERFSTAT Range 1.8739 seconds
2016/04/06 08:31:42 Range Scan of 100000 user documents took 1.873920394s
2016/04/06 08:31:42 PERFSTAT Range 0.0340 seconds
2016/04/06 08:31:42 Range Scan of 100000 user documents took 34.027492ms
2016/04/06 08:31:43 PERFSTAT Range 1.3485 seconds
2016/04/06 08:31:43 Range Scan of 100000 user documents took 1.348600897s
2016/04/06 08:31:46 PERFSTAT Range 3.5384 seconds
2016/04/06 08:31:46 Range Scan of 100000 user documents took 3.538503865s
2016/04/06 08:31:46 PERFSTAT Range 0.0127 seconds
2016/04/06 08:31:46 Range Scan of 100000 user documents took 12.743506ms
2016/04/06 08:31:46 PERFSTAT Range 0.0254 seconds
2016/04/06 08:31:46 Range Scan of 100000 user documents took 25.431081ms
2016/04/06 08:31:46 PERFSTAT Range 0.0210 seconds
2016/04/06 08:31:46 Range Scan of 100000 user documents took 21.086036ms
2016/04/06 08:31:46 PERFSTAT Range 0.0505 seconds
2016/04/06 08:31:46 Range Scan of 100000 user documents took 52.640398ms
2016/04/06 08:31:46 PERFSTAT Range 0.0011 seconds
2016/04/06 08:31:46 Range Scan of 100000 user documents took 1.158909ms
2016/04/06 08:31:46 PERFSTAT Range 0.0843 seconds
2016/04/06 08:31:46 Range Scan of 100000 user documents took 86.892857ms
2016/04/06 08:31:49 PERFSTAT Range 6.9964 seconds
2016/04/06 08:31:49 Range Scan of 100000 user documents took 6.996451825s
2016/04/06 08:31:49 PERFSTAT Range 6.5608 seconds
2016/04/06 08:31:49 Range Scan of 100000 user documents took 6.56084675s
2016/04/06 08:31:49 PERFSTAT Range 0.0149 seconds
2016/04/06 08:31:49 Range Scan of 100000 user documents took 14.929394ms
2016/04/06 08:31:49 PERFSTAT Range 0.0517 seconds
2016/04/06 08:31:49 Range Scan of 100000 user documents took 51.777314ms
2016/04/06 08:31:49 PERFSTAT Range 0.0027 seconds
2016/04/06 08:31:49 Range Scan of 100000 user documents took 2.745113ms
2016/04/06 08:31:49 PERFSTAT Range 0.0008 seconds
2016/04/06 08:31:49 Range Scan of 100000 user documents took 829.036µs
2016/04/06 08:31:49 PERFSTAT Range 13.9745 seconds
2016/04/06 08:31:49 Range Scan of 100000 user documents took 13.974609702s
2016/04/06 08:31:49 PERFSTAT Range 0.0095 seconds
2016/04/06 08:31:49 Range Scan of 100000 user documents took 9.538811ms
2016/04/06 08:31:49 PERFSTAT Range 0.0103 seconds
2016/04/06 08:31:49 Range Scan of 100000 user documents took 10.39597ms
2016/04/06 08:31:49 PERFSTAT Range 7.7836 seconds
2016/04/06 08:31:49 Range Scan of 100000 user documents took 7.783630828s
2016/04/06 08:31:49 PERFSTAT Range 2.9996 seconds
2016/04/06 08:31:49 Range Scan of 100000 user documents took 3.012581848s
2016/04/06 08:31:49 PERFSTAT Range 0.0974 seconds
2016/04/06 08:31:49 Range Scan of 100000 user documents took 97.490557ms
2016/04/06 08:31:51 PERFSTAT Range 1.7294 seconds
2016/04/06 08:31:51 Range Scan of 100000 user documents took 1.734425215s
2016/04/06 08:31:51 PERFSTAT Range 0.0172 seconds
2016/04/06 08:31:51 Range Scan of 100000 user documents took 17.234996ms
2016/04/06 08:31:51 PERFSTAT Range 13.3693 seconds
2016/04/06 08:31:51 Range Scan of 100000 user documents took 13.369353034s
2016/04/06 08:31:51 PERFSTAT Range 0.0370 seconds
2016/04/06 08:31:51 Range Scan of 100000 user documents took 37.100084ms
2016/04/06 08:31:51 PERFSTAT Range 0.0210 seconds
2016/04/06 08:31:51 Range Scan of 100000 user documents took 21.011155ms
2016/04/06 08:31:52 PERFSTAT Range 3.6736 seconds
2016/04/06 08:31:52 Range Scan of 100000 user documents took 3.689388261s
2016/04/06 08:31:54 PERFSTAT Range 4.4616 seconds
2016/04/06 08:31:54 Range Scan of 100000 user documents took 4.461756666s
2016/04/06 08:31:54 PERFSTAT Range 0.0446 seconds
2016/04/06 08:31:54 Range Scan of 100000 user documents took 44.637382ms
2016/04/06 08:31:54 PERFSTAT Range 0.6791 seconds
2016/04/06 08:31:54 Range Scan of 100000 user documents took 679.637806ms
2016/04/06 08:31:54 PERFSTAT Range 0.0096 seconds
2016/04/06 08:31:54 Range Scan of 100000 user documents took 9.588447ms
2016/04/06 08:31:54 PERFSTAT Range 0.0956 seconds
2016/04/06 08:31:54 Range Scan of 100000 user documents took 95.599881ms
2016/04/06 08:31:55 PERFSTAT Range 5.3544 seconds
2016/04/06 08:31:55 Range Scan of 100000 user documents took 5.35454995s
2016/04/06 08:31:55 PERFSTAT Range 0.0197 seconds
2016/04/06 08:31:55 Range Scan of 100000 user documents took 19.803671ms
2016/04/06 08:31:55 PERFSTAT Range 2.6313 seconds
2016/04/06 08:31:55 Range Scan of 100000 user documents took 2.631353729s
2016/04/06 08:31:55 PERFSTAT Range 0.0005 seconds
2016/04/06 08:31:55 Range Scan of 100000 user documents took 5.372968ms
2016/04/06 08:31:55 PERFSTAT Range 0.0027 seconds
2016/04/06 08:31:55 Range Scan of 100000 user documents took 2.702059ms
2016/04/06 08:31:56 PERFSTAT Range 4.9865 seconds
2016/04/06 08:31:56 Range Scan of 100000 user documents took 4.986553861s
2016/04/06 08:31:56 PERFSTAT Range 0.0005 seconds
2016/04/06 08:31:56 Range Scan of 100000 user documents took 582.378µs
2016/04/06 08:31:56 PERFSTAT Range 6.9775 seconds
2016/04/06 08:31:56 Range Scan of 100000 user documents took 6.977546277s
2016/04/06 08:31:56 PERFSTAT Range 0.0058 seconds
2016/04/06 08:31:56 Range Scan of 100000 user documents took 5.870885ms
2016/04/06 08:31:56 PERFSTAT Range 0.0228 seconds
2016/04/06 08:31:56 Range Scan of 100000 user documents took 22.849416ms
2016/04/06 08:31:56 PERFSTAT Range 0.0113 seconds
2016/04/06 08:31:56 Range Scan of 100000 user documents took 11.377982ms
2016/04/06 08:31:58 PERFSTAT Range 2.2293 seconds
2016/04/06 08:31:58 Range Scan of 100000 user documents took 2.229383837s
2016/04/06 08:31:59 PERFSTAT Range 7.2632 seconds
2016/04/06 08:31:59 Range Scan of 100000 user documents took 7.263292987s
2016/04/06 08:32:01 PERFSTAT Range 6.1245 seconds
2016/04/06 08:32:01 Range Scan of 100000 user documents took 6.124608605s
2016/04/06 08:32:01 PERFSTAT Range 5.3567 seconds
2016/04/06 08:32:01 Range Scan of 100000 user documents took 5.356729982s
2016/04/06 08:32:02 PERFSTAT Range 7.1489 seconds
2016/04/06 08:32:02 Range Scan of 100000 user documents took 7.148948303s
2016/04/06 08:32:02 PERFSTAT Range 6.8625 seconds
2016/04/06 08:32:02 Range Scan of 100000 user documents took 6.86249744s
--- PASS: TestConcurrentScans_SameIndex (208.63s)
=== RUN   TestConcurrentScans_MultipleIndexes
2016/04/06 08:32:02 In TestConcurrentScans_MultipleIndexes()
2016/04/06 08:32:02 Generating JSON docs
2016/04/06 08:32:09 Setting initial JSON docs in KV
2016/04/06 08:33:38 All indexers are active
2016/04/06 08:33:38 Creating multiple indexes
2016/04/06 08:33:38 Index found:  index_company
2016/04/06 08:34:29 Created the secondary index index_age. Waiting for it become active
2016/04/06 08:34:29 Index state is INDEX_STATE_ACTIVE
2016/04/06 08:34:29 Index is now active
2016/04/06 08:34:29 PERFSTAT CreateAndBuildIndex 51.1095 seconds
2016/04/06 08:35:15 Created the secondary index index_firstname. Waiting for it become active
2016/04/06 08:35:15 Index state is INDEX_STATE_ACTIVE
2016/04/06 08:35:15 Index is now active
2016/04/06 08:35:15 PERFSTAT CreateAndBuildIndex 45.8765 seconds
2016/04/06 08:35:15 In Range Scan
2016/04/06 08:35:15 In Range Scan
2016/04/06 08:35:15 In Range Scan
2016/04/06 08:35:15 PERFSTAT Range 0.0026 seconds
2016/04/06 08:35:15 PERFSTAT Range 0.0098 seconds
2016/04/06 08:35:15 Range Scan of 100000 user documents took 9.848718ms
2016/04/06 08:35:16 PERFSTAT Range 0.7069 seconds
2016/04/06 08:35:16 Range Scan of 100000 user documents took 706.953145ms
2016/04/06 08:35:17 PERFSTAT Range 0.8765 seconds
2016/04/06 08:35:17 Range Scan of 100000 user documents took 876.5129ms
2016/04/06 08:35:17 PERFSTAT Range 1.9910 seconds
2016/04/06 08:35:17 Range Scan of 100000 user documents took 1.991113449s
2016/04/06 08:35:18 PERFSTAT Range 0.1614 seconds
2016/04/06 08:35:18 Range Scan of 100000 user documents took 161.570243ms
2016/04/06 08:35:18 PERFSTAT Range 0.0005 seconds
2016/04/06 08:35:18 Range Scan of 100000 user documents took 589.837µs
2016/04/06 08:35:22 PERFSTAT Range 5.0191 seconds
2016/04/06 08:35:22 Range Scan of 100000 user documents took 5.019111575s
2016/04/06 08:35:25 PERFSTAT Range 2.5270 seconds
2016/04/06 08:35:25 Range Scan of 100000 user documents took 2.527031428s
2016/04/06 08:35:25 PERFSTAT Range 0.0019 seconds
2016/04/06 08:35:25 Range Scan of 100000 user documents took 1.951336ms
2016/04/06 08:35:25 PERFSTAT Range 0.0005 seconds
2016/04/06 08:35:25 Range Scan of 100000 user documents took 558.421µs
2016/04/06 08:35:25 PERFSTAT Range 9.2382 seconds
2016/04/06 08:35:25 PERFSTAT Range 0.1752 seconds
2016/04/06 08:35:25 Range Scan of 100000 user documents took 175.203644ms
2016/04/06 08:35:25 PERFSTAT Range 7.2479 seconds
2016/04/06 08:35:25 Range Scan of 100000 user documents took 7.247904342s
2016/04/06 08:35:28 PERFSTAT Range 3.5258 seconds
2016/04/06 08:35:28 Range Scan of 100000 user documents took 3.525960313s
2016/04/06 08:35:28 PERFSTAT Range 0.0017 seconds
2016/04/06 08:35:28 Range Scan of 100000 user documents took 1.78789ms
2016/04/06 08:35:31 PERFSTAT Range 5.9601 seconds
2016/04/06 08:35:31 Range Scan of 100000 user documents took 5.960156117s
2016/04/06 08:35:31 PERFSTAT Range 0.0006 seconds
2016/04/06 08:35:31 Range Scan of 100000 user documents took 597.462µs
2016/04/06 08:35:31 PERFSTAT Range 6.8719 seconds
2016/04/06 08:35:31 PERFSTAT Range 0.0005 seconds
2016/04/06 08:35:31 PERFSTAT Range 0.0004 seconds
2016/04/06 08:35:33 PERFSTAT Range 1.7761 seconds
2016/04/06 08:35:33 Range Scan of 100000 user documents took 1.776189647s
2016/04/06 08:35:35 PERFSTAT Range 3.2980 seconds
2016/04/06 08:35:35 PERFSTAT Range 0.0006 seconds
2016/04/06 08:35:35 PERFSTAT Range 0.0009 seconds
2016/04/06 08:35:35 PERFSTAT Range 0.0004 seconds
2016/04/06 08:35:35 PERFSTAT Range 0.0005 seconds
2016/04/06 08:35:35 PERFSTAT Range 0.0012 seconds
2016/04/06 08:35:35 PERFSTAT Range 0.0005 seconds
2016/04/06 08:35:35 PERFSTAT Range 0.0005 seconds
2016/04/06 08:35:35 PERFSTAT Range 0.0005 seconds
2016/04/06 08:35:35 PERFSTAT Range 0.0005 seconds
2016/04/06 08:35:36 PERFSTAT Range 7.9783 seconds
2016/04/06 08:35:36 Range Scan of 100000 user documents took 7.978383015s
2016/04/06 08:35:36 PERFSTAT Range 0.0008 seconds
2016/04/06 08:35:36 Range Scan of 100000 user documents took 842.792µs
2016/04/06 08:35:37 PERFSTAT Range 0.4896 seconds
2016/04/06 08:35:37 Range Scan of 100000 user documents took 489.6365ms
2016/04/06 08:35:37 PERFSTAT Range 0.0005 seconds
2016/04/06 08:35:37 Range Scan of 100000 user documents took 592.052µs
2016/04/06 08:35:40 PERFSTAT Range 4.9376 seconds
2016/04/06 08:35:40 PERFSTAT Range 0.0007 seconds
2016/04/06 08:35:41 PERFSTAT Range 8.1903 seconds
2016/04/06 08:35:41 Range Scan of 100000 user documents took 8.190494746s
2016/04/06 08:35:44 PERFSTAT Range 4.1755 seconds
2016/04/06 08:35:44 PERFSTAT Range 0.0006 seconds
2016/04/06 08:35:45 PERFSTAT Range 8.3715 seconds
2016/04/06 08:35:45 Range Scan of 100000 user documents took 8.371500946s
2016/04/06 08:35:48 PERFSTAT Range 6.8547 seconds
2016/04/06 08:35:48 Range Scan of 100000 user documents took 6.854806561s
2016/04/06 08:35:50 PERFSTAT Range 5.9375 seconds
2016/04/06 08:35:50 PERFSTAT Range 0.0005 seconds
2016/04/06 08:35:50 PERFSTAT Range 0.0012 seconds
2016/04/06 08:35:50 PERFSTAT Range 0.0005 seconds
2016/04/06 08:35:51 PERFSTAT Range 5.6922 seconds
2016/04/06 08:35:51 Range Scan of 100000 user documents took 5.692333927s
2016/04/06 08:35:51 PERFSTAT Range 1.3438 seconds
2016/04/06 08:35:51 PERFSTAT Range 0.0014 seconds
2016/04/06 08:35:52 PERFSTAT Range 0.8225 seconds
2016/04/06 08:35:52 PERFSTAT Range 0.0011 seconds
2016/04/06 08:35:52 PERFSTAT Range 0.0025 seconds
2016/04/06 08:35:52 PERFSTAT Range 0.0004 seconds
2016/04/06 08:35:52 PERFSTAT Range 0.0004 seconds
2016/04/06 08:35:52 PERFSTAT Range 0.0004 seconds
2016/04/06 08:35:52 PERFSTAT Range 1.2750 seconds
2016/04/06 08:35:52 Range Scan of 100000 user documents took 1.275064323s
2016/04/06 08:35:52 PERFSTAT Range 0.0004 seconds
2016/04/06 08:35:52 Range Scan of 100000 user documents took 466.724µs
2016/04/06 08:35:52 PERFSTAT Range 0.0004 seconds
2016/04/06 08:35:52 Range Scan of 100000 user documents took 408.742µs
2016/04/06 08:35:52 PERFSTAT Range 0.0004 seconds
2016/04/06 08:35:52 Range Scan of 100000 user documents took 380.722µs
2016/04/06 08:35:56 PERFSTAT Range 8.3110 seconds
2016/04/06 08:35:56 Range Scan of 100000 user documents took 8.311109655s
2016/04/06 08:35:56 PERFSTAT Range 0.0017 seconds
2016/04/06 08:35:56 Range Scan of 100000 user documents took 1.737941ms
2016/04/06 08:35:56 PERFSTAT Range 0.0005 seconds
2016/04/06 08:35:56 Range Scan of 100000 user documents took 524.683µs
2016/04/06 08:35:56 PERFSTAT Range 0.0006 seconds
2016/04/06 08:35:56 Range Scan of 100000 user documents took 608.001µs
2016/04/06 08:35:56 PERFSTAT Range 0.0004 seconds
2016/04/06 08:35:56 Range Scan of 100000 user documents took 470.735µs
2016/04/06 08:35:58 PERFSTAT Range 1.9210 seconds
2016/04/06 08:35:58 Range Scan of 100000 user documents took 1.921106745s
2016/04/06 08:35:58 PERFSTAT Range 6.4667 seconds
2016/04/06 08:36:00 PERFSTAT Range 8.3196 seconds
2016/04/06 08:36:00 Range Scan of 100000 user documents took 8.31965088s
2016/04/06 08:36:00 PERFSTAT Range 0.0008 seconds
2016/04/06 08:36:00 Range Scan of 100000 user documents took 820.846µs
2016/04/06 08:36:00 PERFSTAT Range 0.0021 seconds
2016/04/06 08:36:00 Range Scan of 100000 user documents took 2.154883ms
2016/04/06 08:36:00 PERFSTAT Range 0.0005 seconds
2016/04/06 08:36:00 Range Scan of 100000 user documents took 519.663µs
2016/04/06 08:36:00 PERFSTAT Range 0.0004 seconds
2016/04/06 08:36:00 Range Scan of 100000 user documents took 470.26µs
2016/04/06 08:36:00 PERFSTAT Range 0.0003 seconds
2016/04/06 08:36:00 Range Scan of 100000 user documents took 363.327µs
2016/04/06 08:36:00 PERFSTAT Range 0.0003 seconds
2016/04/06 08:36:00 Range Scan of 100000 user documents took 349.147µs
2016/04/06 08:36:00 PERFSTAT Range 0.0004 seconds
2016/04/06 08:36:00 Range Scan of 100000 user documents took 412.901µs
2016/04/06 08:36:03 PERFSTAT Range 4.3709 seconds
2016/04/06 08:36:04 PERFSTAT Range 6.1192 seconds
2016/04/06 08:36:04 Range Scan of 100000 user documents took 6.119282864s
2016/04/06 08:36:04 PERFSTAT Range 0.0005 seconds
2016/04/06 08:36:04 Range Scan of 100000 user documents took 613.368µs
2016/04/06 08:36:05 PERFSTAT Range 1.3166 seconds
2016/04/06 08:36:05 Range Scan of 100000 user documents took 1.316664595s
2016/04/06 08:36:07 PERFSTAT Range 4.2986 seconds
2016/04/06 08:36:08 PERFSTAT Range 3.0047 seconds
2016/04/06 08:36:08 Range Scan of 100000 user documents took 3.004828132s
2016/04/06 08:36:08 PERFSTAT Range 0.0006 seconds
2016/04/06 08:36:08 Range Scan of 100000 user documents took 645.189µs
2016/04/06 08:36:08 PERFSTAT Range 8.0698 seconds
2016/04/06 08:36:08 Range Scan of 100000 user documents took 8.069865s
2016/04/06 08:36:10 PERFSTAT Range 1.9011 seconds
2016/04/06 08:36:10 Range Scan of 100000 user documents took 1.901124735s
2016/04/06 08:36:10 PERFSTAT Range 0.0005 seconds
2016/04/06 08:36:10 Range Scan of 100000 user documents took 518.058µs
2016/04/06 08:36:10 PERFSTAT Range 0.0004 seconds
2016/04/06 08:36:10 Range Scan of 100000 user documents took 429.286µs
2016/04/06 08:36:10 PERFSTAT Range 0.0008 seconds
2016/04/06 08:36:10 Range Scan of 100000 user documents took 817.707µs
2016/04/06 08:36:10 PERFSTAT Range 0.0004 seconds
2016/04/06 08:36:10 Range Scan of 100000 user documents took 432.672µs
2016/04/06 08:36:12 PERFSTAT Range 1.7750 seconds
2016/04/06 08:36:12 Range Scan of 100000 user documents took 1.775079917s
2016/04/06 08:36:14 PERFSTAT Range 6.6642 seconds
2016/04/06 08:36:15 PERFSTAT Range 3.1944 seconds
2016/04/06 08:36:15 Range Scan of 100000 user documents took 3.194465423s
2016/04/06 08:36:15 PERFSTAT Range 0.0009 seconds
2016/04/06 08:36:15 Range Scan of 100000 user documents took 895.21µs
2016/04/06 08:36:17 PERFSTAT Range 8.3605 seconds
2016/04/06 08:36:17 Range Scan of 100000 user documents took 8.36059054s
2016/04/06 08:36:17 PERFSTAT Range 0.0005 seconds
2016/04/06 08:36:17 Range Scan of 100000 user documents took 534.8µs
2016/04/06 08:36:21 PERFSTAT Range 6.9564 seconds
2016/04/06 08:36:23 PERFSTAT Range 7.4369 seconds
2016/04/06 08:36:23 Range Scan of 100000 user documents took 7.43695156s
2016/04/06 08:36:26 PERFSTAT Range 9.4333 seconds
2016/04/06 08:36:26 Range Scan of 100000 user documents took 9.441458469s
2016/04/06 08:36:26 PERFSTAT Range 5.6791 seconds
2016/04/06 08:36:26 PERFSTAT Range 0.0005 seconds
2016/04/06 08:36:26 PERFSTAT Range 0.0004 seconds
2016/04/06 08:36:26 PERFSTAT Range 0.0004 seconds
2016/04/06 08:36:26 PERFSTAT Range 0.0004 seconds
2016/04/06 08:36:26 PERFSTAT Range 0.0004 seconds
2016/04/06 08:36:26 PERFSTAT Range 0.0004 seconds
2016/04/06 08:36:26 PERFSTAT Range 3.7647 seconds
2016/04/06 08:36:26 Range Scan of 100000 user documents took 3.764753527s
2016/04/06 08:36:26 PERFSTAT Range 0.0005 seconds
2016/04/06 08:36:26 Range Scan of 100000 user documents took 508.837µs
2016/04/06 08:36:30 PERFSTAT Range 3.9926 seconds
2016/04/06 08:36:30 Range Scan of 100000 user documents took 3.993424022s
2016/04/06 08:36:30 PERFSTAT Range 0.0005 seconds
2016/04/06 08:36:30 Range Scan of 100000 user documents took 557.058µs
2016/04/06 08:36:34 PERFSTAT Range 7.7890 seconds
2016/04/06 08:36:34 Range Scan of 100000 user documents took 7.789065266s
2016/04/06 08:36:34 PERFSTAT Range 0.0005 seconds
2016/04/06 08:36:34 Range Scan of 100000 user documents took 504.333µs
2016/04/06 08:36:34 PERFSTAT Range 3.8334 seconds
2016/04/06 08:36:34 Range Scan of 100000 user documents took 3.833500858s
2016/04/06 08:36:34 PERFSTAT Range 0.0005 seconds
2016/04/06 08:36:34 Range Scan of 100000 user documents took 567.931µs
2016/04/06 08:36:36 PERFSTAT Range 9.1301 seconds
2016/04/06 08:36:36 PERFSTAT Range 1.8520 seconds
2016/04/06 08:36:36 Range Scan of 100000 user documents took 1.852089908s
2016/04/06 08:36:39 PERFSTAT Range 2.9980 seconds
2016/04/06 08:36:39 Range Scan of 100000 user documents took 2.99807684s
2016/04/06 08:36:39 PERFSTAT Range 0.0005 seconds
2016/04/06 08:36:39 Range Scan of 100000 user documents took 558.511µs
2016/04/06 08:36:39 PERFSTAT Range 0.0004 seconds
2016/04/06 08:36:39 Range Scan of 100000 user documents took 395.713µs
2016/04/06 08:36:39 PERFSTAT Range 0.0003 seconds
2016/04/06 08:36:39 Range Scan of 100000 user documents took 339.951µs
2016/04/06 08:36:40 PERFSTAT Range 6.1490 seconds
2016/04/06 08:36:40 Range Scan of 100000 user documents took 6.149151635s
2016/04/06 08:36:42 PERFSTAT Range 2.9645 seconds
2016/04/06 08:36:42 Range Scan of 100000 user documents took 2.964604447s
2016/04/06 08:36:43 PERFSTAT Range 2.7191 seconds
2016/04/06 08:36:43 Range Scan of 100000 user documents took 2.719152712s
2016/04/06 08:36:43 PERFSTAT Range 0.0005 seconds
2016/04/06 08:36:43 Range Scan of 100000 user documents took 583.253µs
2016/04/06 08:36:44 PERFSTAT Range 1.9806 seconds
2016/04/06 08:36:44 Range Scan of 100000 user documents took 1.980785568s
2016/04/06 08:36:44 PERFSTAT Range 0.0006 seconds
2016/04/06 08:36:44 Range Scan of 100000 user documents took 642.862µs
2016/04/06 08:36:44 PERFSTAT Range 8.4199 seconds
2016/04/06 08:36:46 PERFSTAT Range 2.4130 seconds
2016/04/06 08:36:46 Range Scan of 100000 user documents took 2.413116008s
2016/04/06 08:36:46 PERFSTAT Range 0.0012 seconds
2016/04/06 08:36:46 Range Scan of 100000 user documents took 1.226545ms
2016/04/06 08:36:46 PERFSTAT Range 0.0005 seconds
2016/04/06 08:36:46 Range Scan of 100000 user documents took 513.101µs
2016/04/06 08:36:48 PERFSTAT Range 4.6806 seconds
2016/04/06 08:36:48 Range Scan of 100000 user documents took 4.680629508s
2016/04/06 08:36:48 PERFSTAT Range 0.0005 seconds
2016/04/06 08:36:48 Range Scan of 100000 user documents took 612.365µs
2016/04/06 08:36:48 PERFSTAT Range 0.0033 seconds
2016/04/06 08:36:48 Range Scan of 100000 user documents took 3.37572ms
2016/04/06 08:36:48 PERFSTAT Range 0.1903 seconds
2016/04/06 08:36:48 Range Scan of 100000 user documents took 190.330249ms
2016/04/06 08:36:49 PERFSTAT Range 4.8039 seconds
2016/04/06 08:36:50 PERFSTAT Range 3.6029 seconds
2016/04/06 08:36:50 Range Scan of 100000 user documents took 3.604186791s
2016/04/06 08:36:50 PERFSTAT Range 0.1318 seconds
2016/04/06 08:36:50 Range Scan of 100000 user documents took 131.891009ms
2016/04/06 08:36:50 PERFSTAT Range 0.0012 seconds
2016/04/06 08:36:50 Range Scan of 100000 user documents took 1.342453ms
2016/04/06 08:36:50 PERFSTAT Range 0.0121 seconds
2016/04/06 08:36:50 Range Scan of 100000 user documents took 71.904559ms
2016/04/06 08:36:53 PERFSTAT Range 3.0618 seconds
2016/04/06 08:36:53 Range Scan of 100000 user documents took 3.061883693s
2016/04/06 08:36:53 PERFSTAT Range 0.0005 seconds
2016/04/06 08:36:53 Range Scan of 100000 user documents took 547.29µs
2016/04/06 08:36:53 PERFSTAT Range 0.0004 seconds
2016/04/06 08:36:53 Range Scan of 100000 user documents took 505.352µs
2016/04/06 08:36:53 PERFSTAT Range 4.5778 seconds
2016/04/06 08:36:53 PERFSTAT Range 0.0006 seconds
2016/04/06 08:36:56 PERFSTAT Range 8.4250 seconds
2016/04/06 08:36:56 Range Scan of 100000 user documents took 8.4251089s
2016/04/06 08:36:56 PERFSTAT Range 0.0006 seconds
2016/04/06 08:36:56 Range Scan of 100000 user documents took 588.797µs
2016/04/06 08:36:56 PERFSTAT Range 0.0004 seconds
2016/04/06 08:36:56 Range Scan of 100000 user documents took 465.86µs
2016/04/06 08:36:57 PERFSTAT Range 3.7661 seconds
2016/04/06 08:36:57 Range Scan of 100000 user documents took 3.766101457s
2016/04/06 08:36:57 PERFSTAT Range 0.0006 seconds
2016/04/06 08:36:57 Range Scan of 100000 user documents took 659.986µs
2016/04/06 08:36:58 PERFSTAT Range 4.2237 seconds
2016/04/06 08:36:58 PERFSTAT Range 0.0006 seconds
2016/04/06 08:36:58 PERFSTAT Range 0.0033 seconds
2016/04/06 08:36:58 PERFSTAT Range 0.0004 seconds
2016/04/06 08:36:59 PERFSTAT Range 2.1640 seconds
2016/04/06 08:36:59 Range Scan of 100000 user documents took 2.164147744s
2016/04/06 08:36:59 PERFSTAT Range 1.5634 seconds
2016/04/06 08:36:59 PERFSTAT Range 0.0005 seconds
2016/04/06 08:37:00 PERFSTAT Range 0.5790 seconds
2016/04/06 08:37:02 PERFSTAT Range 3.0192 seconds
2016/04/06 08:37:02 Range Scan of 100000 user documents took 3.019272176s
2016/04/06 08:37:02 PERFSTAT Range 0.0018 seconds
2016/04/06 08:37:02 Range Scan of 100000 user documents took 1.828098ms
2016/04/06 08:37:02 PERFSTAT Range 5.0280 seconds
2016/04/06 08:37:02 Range Scan of 100000 user documents took 5.028168529s
2016/04/06 08:37:02 PERFSTAT Range 0.0005 seconds
2016/04/06 08:37:02 Range Scan of 100000 user documents took 578.588µs
2016/04/06 08:37:03 PERFSTAT Range 1.6645 seconds
2016/04/06 08:37:03 Range Scan of 100000 user documents took 1.664595913s
2016/04/06 08:37:07 PERFSTAT Range 6.7754 seconds
2016/04/06 08:37:07 PERFSTAT Range 0.0006 seconds
2016/04/06 08:37:07 PERFSTAT Range 0.0004 seconds
2016/04/06 08:37:07 PERFSTAT Range 0.0004 seconds
2016/04/06 08:37:07 PERFSTAT Range 0.0004 seconds
2016/04/06 08:37:07 PERFSTAT Range 5.6484 seconds
2016/04/06 08:37:07 Range Scan of 100000 user documents took 5.64847394s
2016/04/06 08:37:07 PERFSTAT Range 0.0013 seconds
2016/04/06 08:37:07 Range Scan of 100000 user documents took 1.424169ms
2016/04/06 08:37:08 PERFSTAT Range 4.7032 seconds
2016/04/06 08:37:08 Range Scan of 100000 user documents took 4.703334633s
2016/04/06 08:37:08 PERFSTAT Range 0.0009 seconds
2016/04/06 08:37:08 Range Scan of 100000 user documents took 979.466µs
2016/04/06 08:37:08 PERFSTAT Range 1.5640 seconds
2016/04/06 08:37:08 PERFSTAT Range 0.0006 seconds
2016/04/06 08:37:08 PERFSTAT Range 0.0014 seconds
2016/04/06 08:37:08 PERFSTAT Range 0.0005 seconds
2016/04/06 08:37:08 PERFSTAT Range 0.0011 seconds
2016/04/06 08:37:08 PERFSTAT Range 0.0004 seconds
2016/04/06 08:37:09 PERFSTAT Range 1.3342 seconds
2016/04/06 08:37:10 PERFSTAT Range 2.6865 seconds
2016/04/06 08:37:10 Range Scan of 100000 user documents took 2.686592758s
2016/04/06 08:37:11 PERFSTAT Range 2.4189 seconds
2016/04/06 08:37:11 Range Scan of 100000 user documents took 2.419055969s
2016/04/06 08:37:11 PERFSTAT Range 0.8784 seconds
2016/04/06 08:37:11 Range Scan of 100000 user documents took 878.493933ms
2016/04/06 08:37:12 PERFSTAT Range 1.0059 seconds
2016/04/06 08:37:12 Range Scan of 100000 user documents took 1.006026366s
2016/04/06 08:37:12 PERFSTAT Range 0.0044 seconds
2016/04/06 08:37:12 Range Scan of 100000 user documents took 4.440615ms
2016/04/06 08:37:12 PERFSTAT Range 0.0005 seconds
2016/04/06 08:37:12 Range Scan of 100000 user documents took 554.544µs
2016/04/06 08:37:17 PERFSTAT Range 7.8942 seconds
2016/04/06 08:37:19 PERFSTAT Range 7.7659 seconds
2016/04/06 08:37:19 Range Scan of 100000 user documents took 7.765916584s
2016/04/06 08:37:19 PERFSTAT Range 7.8061 seconds
2016/04/06 08:37:19 Range Scan of 100000 user documents took 7.806131037s
--- PASS: TestConcurrentScans_MultipleIndexes (317.71s)
=== RUN   TestMutationsWithMultipleIndexBuilds
2016/04/06 08:37:19 In TestMutationsWithMultipleIndexBuilds()
2016/04/06 08:37:19 In DropAllSecondaryIndexes()
2016/04/06 08:37:19 Dropped index index_age
2016/04/06 08:37:19 Dropped index index_firstname
2016/04/06 08:37:19 Dropped index index_company
2016/04/06 08:37:19 Generating JSON docs
2016/04/06 08:37:27 Setting initial JSON docs in KV
2016/04/06 08:38:53 Created the secondary index index_primary. Waiting for it become active
2016/04/06 08:38:53 Index state is INDEX_STATE_ACTIVE
2016/04/06 08:38:53 Index is now active
2016/04/06 08:38:53 PERFSTAT CreateAndBuildIndex 41.6698 seconds
2016/04/06 08:38:53 Creating multiple indexes in deferred mode
2016/04/06 08:38:53 Created the secondary index index_company
2016/04/06 08:38:53 Created the secondary index index_age
2016/04/06 08:38:53 Created the secondary index index_firstname
2016/04/06 08:38:53 Created the secondary index index_lastname
2016/04/06 08:38:53 Build Indexes and wait for indexes to become active: [index_company index_age index_firstname index_lastname]
2016/04/06 08:38:53 Build command issued for the deferred indexes [index_company index_age index_firstname index_lastname]
2016/04/06 08:38:53 Waiting for the index index_company to become active
2016/04/06 08:38:53 Index state is INDEX_STATE_READY
2016/04/06 08:38:54 Index state is INDEX_STATE_INITIAL
2016/04/06 08:38:55 Index state is INDEX_STATE_INITIAL
2016/04/06 08:38:56 Index state is INDEX_STATE_INITIAL
2016/04/06 08:38:57 Index state is INDEX_STATE_INITIAL
2016/04/06 08:38:58 Index state is INDEX_STATE_INITIAL
2016/04/06 08:38:59 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:00 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:01 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:02 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:03 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:04 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:05 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:06 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:07 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:08 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:09 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:10 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:11 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:12 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:13 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:14 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:15 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:16 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:17 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:18 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:19 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:20 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:21 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:22 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:23 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:24 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:25 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:26 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:27 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:28 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:29 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:30 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:31 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:32 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:33 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:34 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:35 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:36 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:37 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:38 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:39 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:40 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:41 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:42 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:43 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:44 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:45 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:46 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:47 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:48 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:49 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:50 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:51 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:52 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:53 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:54 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:55 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:56 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:57 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:58 Index state is INDEX_STATE_INITIAL
2016/04/06 08:39:59 Index state is INDEX_STATE_INITIAL
2016/04/06 08:40:00 Index state is INDEX_STATE_INITIAL
2016/04/06 08:40:01 Index state is INDEX_STATE_INITIAL
2016/04/06 08:40:02 Index state is INDEX_STATE_INITIAL
2016/04/06 08:40:03 Index state is INDEX_STATE_INITIAL
2016/04/06 08:40:04 Index state is INDEX_STATE_INITIAL
2016/04/06 08:40:05 Index state is INDEX_STATE_INITIAL
2016/04/06 08:40:06 Index state is INDEX_STATE_INITIAL
2016/04/06 08:40:07 Index state is INDEX_STATE_INITIAL
2016/04/06 08:40:08 Index state is INDEX_STATE_INITIAL
2016/04/06 08:40:09 Index state is INDEX_STATE_INITIAL
2016/04/06 08:40:10 Index state is INDEX_STATE_INITIAL
2016/04/06 08:40:11 Index state is INDEX_STATE_INITIAL
2016/04/06 08:40:12 Index state is INDEX_STATE_INITIAL
2016/04/06 08:40:13 Index state is INDEX_STATE_INITIAL
2016/04/06 08:40:14 Index state is INDEX_STATE_INITIAL
2016/04/06 08:40:15 Index state is INDEX_STATE_INITIAL
2016/04/06 08:40:16 Index state is INDEX_STATE_INITIAL
2016/04/06 08:40:17 Index state is INDEX_STATE_INITIAL
2016/04/06 08:40:18 Index state is INDEX_STATE_INITIAL
2016/04/06 08:40:19 Index state is INDEX_STATE_INITIAL
2016/04/06 08:40:20 Index state is INDEX_STATE_INITIAL
2016/04/06 08:40:21 Index state is INDEX_STATE_INITIAL
2016/04/06 08:40:22 Index state is INDEX_STATE_INITIAL
2016/04/06 08:40:23 Index state is INDEX_STATE_INITIAL
2016/04/06 08:40:24 Index state is INDEX_STATE_INITIAL
2016/04/06 08:40:25 Index state is INDEX_STATE_INITIAL
2016/04/06 08:40:26 Index state is INDEX_STATE_INITIAL
2016/04/06 08:40:27 Index state is INDEX_STATE_INITIAL
2016/04/06 08:40:28 Index state is INDEX_STATE_INITIAL
2016/04/06 08:40:29 Index state is INDEX_STATE_INITIAL
2016/04/06 08:40:30 Index state is INDEX_STATE_INITIAL
2016/04/06 08:40:31 Index state is INDEX_STATE_INITIAL
2016/04/06 08:40:32 Index state is INDEX_STATE_INITIAL
2016/04/06 08:40:33 Index state is INDEX_STATE_INITIAL
2016/04/06 08:40:34 Index state is INDEX_STATE_INITIAL
2016/04/06 08:40:35 Index state is INDEX_STATE_INITIAL
2016/04/06 08:40:36 Index state is INDEX_STATE_INITIAL
2016/04/06 08:40:37 Index state is INDEX_STATE_ACTIVE
2016/04/06 08:40:37 Index is now active
2016/04/06 08:40:37 Waiting for the index index_age to become active
2016/04/06 08:40:37 Index state is INDEX_STATE_ACTIVE
2016/04/06 08:40:37 Index is now active
2016/04/06 08:40:37 Waiting for the index index_firstname to become active
2016/04/06 08:40:37 Index state is INDEX_STATE_ACTIVE
2016/04/06 08:40:37 Index is now active
2016/04/06 08:40:37 Waiting for the index index_lastname to become active
2016/04/06 08:40:37 Index state is INDEX_STATE_ACTIVE
2016/04/06 08:40:37 Index is now active
--- PASS: TestMutationsWithMultipleIndexBuilds (198.14s)
PASS
ok  	github.com/couchbase/indexing/secondary/tests/largedatatests	1205.068s

Integration tests

echo "Running 2i integration tests with 4 node cluster"
Running 2i integration tests with 4 node cluster
scripts/start_cluster_and_run_tests.sh b/resources/dev-4-nodes-xdcr_n1ql_2i.ini conf/simple_2i_n1ql.conf
/opt/build/testrunner /opt/build/testrunner
make[1]: Entering directory '/opt/build/ns_server'
cd build && make --no-print-directory ns_dataclean
Built target ns_dataclean
make[1]: Leaving directory '/opt/build/ns_server'
make[1]: Entering directory '/opt/build/ns_server'
cd build && make --no-print-directory all
Built target all_images
Building Go target vbmap using Go 1.5.2
-- Executing: /home/buildbot/.cbdepscache/exploded/x86_64/go-1.5.2/go/bin/go install -pkgdir /opt/build/build/gopkg/go-1.5.2 -tags="" -gcflags="" -ldflags=""  vbmap
Built target vbmap
Building Go target ns_gozip using Go 1.5.2
-- Executing: /home/buildbot/.cbdepscache/exploded/x86_64/go-1.5.2/go/bin/go install -pkgdir /opt/build/build/gopkg/go-1.5.2 -tags="" -gcflags="" -ldflags=""  gozip
Built target ns_gozip
Building Go target ns_goport using Go 1.5.2
-- Executing: /home/buildbot/.cbdepscache/exploded/x86_64/go-1.5.2/go/bin/go install -pkgdir /opt/build/build/gopkg/go-1.5.2 -tags="" -gcflags="" -ldflags=""  goport
Built target ns_goport
Building Go target ns_generate_cert using Go 1.5.2
-- Executing: /home/buildbot/.cbdepscache/exploded/x86_64/go-1.5.2/go/bin/go install -pkgdir /opt/build/build/gopkg/go-1.5.2 -tags="" -gcflags="" -ldflags=""  generate_cert
Built target ns_generate_cert
Building Go target ns_godu using Go 1.5.2
-- Executing: /home/buildbot/.cbdepscache/exploded/x86_64/go-1.5.2/go/bin/go install -pkgdir /opt/build/build/gopkg/go-1.5.2 -tags="" -gcflags="" -ldflags=""  godu
Built target ns_godu
Building Go target ns_minify using Go 1.5.2
-- Executing: /home/buildbot/.cbdepscache/exploded/x86_64/go-1.5.2/go/bin/go install -pkgdir /opt/build/build/gopkg/go-1.5.2 -tags="" -gcflags="" -ldflags=""  minify
Built target ns_minify
-- Building app.min.js and index.min.html ...
Built target minify
Built target ale-cfg
==> ale (compile)
Built target ale
==> ns_couchdb (compile)
Built target ns_couchdb
==> gen_smtp (compile)
Built target gen_smtp
Built target ns_cfg
==> mlockall (compile)
Built target mlockall
==> ns_babysitter (compile)
Built target ns_babysitter
==> ns_ssl_proxy (compile)
Built target ns_ssl_proxy
==> ns_server (compile)
Built target ns_server
make[1]: Leaving directory '/opt/build/ns_server'
/opt/build/testrunner
filename: conf/simple_2i_n1ql.conf
prefix: 2i.indexscans_2i.SecondaryIndexingScanTests
prefix: 2i.indexcreatedrop_2i.SecondaryIndexingCreateDropTests
prefix: 2i.cluster_ops_2i.SecondaryIndexingClusterOpsTests
Global Test input params:
{'cluster_name': 'dev-4-nodes-xdcr_n1ql_2i',
 'conf_file': 'conf/simple_2i_n1ql.conf',
 'ini': 'b/resources/dev-4-nodes-xdcr_n1ql_2i.ini',
 'makefile': 'True',
 'num_nodes': 4,
 'spec': 'simple_2i_n1ql'}
Logs will be stored at /opt/build/testrunner/logs/testrunner-16-Apr-06_08-41-08/test_1

./testrunner -i b/resources/dev-4-nodes-xdcr_n1ql_2i.ini makefile=True -t 2i.indexscans_2i.SecondaryIndexingScanTests.test_multi_create_query_explain_drop_index,groups=simple:equals:no_orderby_groupby:range,dataset=default,doc-per-day=1,use_gsi_for_primary=True,use_gsi_for_secondary=True

Test Input params:
{'doc-per-day': '1', 'use_gsi_for_primary': 'True', 'use_gsi_for_secondary': 'True', 'makefile': 'True', 'conf_file': 'conf/simple_2i_n1ql.conf', 'dataset': 'default', 'cluster_name': 'dev-4-nodes-xdcr_n1ql_2i', 'ini': 'b/resources/dev-4-nodes-xdcr_n1ql_2i.ini', 'groups': 'simple:equals:no_orderby_groupby:range', 'case_number': 1, 'num_nodes': 4, 'logs_folder': '/opt/build/testrunner/logs/testrunner-16-Apr-06_08-41-08/test_1', 'spec': 'simple_2i_n1ql'}
Run before suite setup for 2i.indexscans_2i.SecondaryIndexingScanTests.test_multi_create_query_explain_drop_index
2016-04-06 08:41:08 | WARNING | MainProcess | MainThread | [data_helper.] Can not import concurrent module. Data for each server will be loaded/retrieved sequentially
test_multi_create_query_explain_drop_index (2i.indexscans_2i.SecondaryIndexingScanTests) ... 2016-04-06 08:41:08 | ERROR | MainProcess | test_thread | [rest_client._http_request] socket error while connecting to http://127.0.0.1:9000/nodes/self error [Errno 111] Connection refused 
2016-04-06 08:41:11 | ERROR | MainProcess | test_thread | [rest_client._http_request] socket error while connecting to http://127.0.0.1:9000/nodes/self error [Errno 111] Connection refused 
2016-04-06 08:41:14 | ERROR | MainProcess | test_thread | [rest_client._http_request] socket error while connecting to http://127.0.0.1:9000/nodes/self error [Errno 111] Connection refused 
2016-04-06 08:41:17 | ERROR | MainProcess | test_thread | [rest_client._http_request] http://127.0.0.1:9000/pools/default error 404 reason: unknown "unknown pool"
2016-04-06 08:41:17 | INFO | MainProcess | test_thread | [rest_client.get_nodes_versions] Node versions in cluster []
2016-04-06 08:41:17 | INFO | MainProcess | test_thread | [basetestcase.setUp] ==============  basetestcase setup was started for test #1 test_multi_create_query_explain_drop_index==============
2016-04-06 08:41:17 | ERROR | MainProcess | test_thread | [rest_client._http_request] http://127.0.0.1:9000/pools/default/ error 404 reason: unknown "unknown pool"
2016-04-06 08:41:17 | INFO | MainProcess | test_thread | [bucket_helper.delete_all_buckets_or_assert] deleting existing buckets [] on 127.0.0.1
2016-04-06 08:41:17 | INFO | MainProcess | test_thread | [bucket_helper.delete_all_buckets_or_assert] deleting existing buckets [] on 127.0.0.1
2016-04-06 08:41:17 | INFO | MainProcess | test_thread | [bucket_helper.delete_all_buckets_or_assert] deleting existing buckets [] on 127.0.0.1
2016-04-06 08:41:17 | INFO | MainProcess | test_thread | [bucket_helper.delete_all_buckets_or_assert] deleting existing buckets [] on 127.0.0.1
2016-04-06 08:41:17 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 10 secs.  ...
2016-04-06 08:41:27 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] waiting for ns_server @ 127.0.0.1:9000
2016-04-06 08:41:27 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] ns_server @ 127.0.0.1:9000 is running
2016-04-06 08:41:27 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] waiting for ns_server @ 127.0.0.1:9001
2016-04-06 08:41:27 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] ns_server @ 127.0.0.1:9001 is running
2016-04-06 08:41:27 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] waiting for ns_server @ 127.0.0.1:9002
2016-04-06 08:41:27 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] ns_server @ 127.0.0.1:9002 is running
2016-04-06 08:41:27 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] waiting for ns_server @ 127.0.0.1:9003
2016-04-06 08:41:27 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] ns_server @ 127.0.0.1:9003 is running
2016-04-06 08:41:27 | ERROR | MainProcess | test_thread | [rest_client._http_request] http://127.0.0.1:9000/pools/default error 404 reason: unknown "unknown pool"
2016-04-06 08:41:27 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] cannot find service node index in cluster 
2016-04-06 08:41:27 | INFO | MainProcess | test_thread | [basetestcase.tearDown] ==============  basetestcase cleanup was started for test #1 test_multi_create_query_explain_drop_index ==============
2016-04-06 08:41:27 | ERROR | MainProcess | test_thread | [rest_client._http_request] http://127.0.0.1:9000/pools/default/ error 404 reason: unknown "unknown pool"
2016-04-06 08:41:27 | INFO | MainProcess | test_thread | [bucket_helper.delete_all_buckets_or_assert] deleting existing buckets [] on 127.0.0.1
2016-04-06 08:41:27 | INFO | MainProcess | test_thread | [bucket_helper.delete_all_buckets_or_assert] deleting existing buckets [] on 127.0.0.1
2016-04-06 08:41:27 | INFO | MainProcess | test_thread | [bucket_helper.delete_all_buckets_or_assert] deleting existing buckets [] on 127.0.0.1
2016-04-06 08:41:27 | INFO | MainProcess | test_thread | [bucket_helper.delete_all_buckets_or_assert] deleting existing buckets [] on 127.0.0.1
2016-04-06 08:41:28 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] waiting for ns_server @ 127.0.0.1:9000
2016-04-06 08:41:28 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] ns_server @ 127.0.0.1:9000 is running
2016-04-06 08:41:28 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] waiting for ns_server @ 127.0.0.1:9001
2016-04-06 08:41:28 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] ns_server @ 127.0.0.1:9001 is running
2016-04-06 08:41:28 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] waiting for ns_server @ 127.0.0.1:9002
2016-04-06 08:41:28 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] ns_server @ 127.0.0.1:9002 is running
2016-04-06 08:41:28 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] waiting for ns_server @ 127.0.0.1:9003
2016-04-06 08:41:28 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] ns_server @ 127.0.0.1:9003 is running
2016-04-06 08:41:28 | INFO | MainProcess | test_thread | [basetestcase.tearDown] ==============  basetestcase cleanup was finished for test #1 test_multi_create_query_explain_drop_index ==============
Cluster instance shutdown with force
2016-04-06 08:41:28 | INFO | MainProcess | test_thread | [basetestcase.setUp] initializing cluster
2016-04-06 08:41:29 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster_memoryQuota] pools/default params : memoryQuota=8041
2016-04-06 08:41:29 | INFO | MainProcess | Cluster_Thread | [rest_client.set_indexer_storage_mode] settings/indexes params : storageMode=forestdb
2016-04-06 08:41:29 | INFO | MainProcess | Cluster_Thread | [rest_client.init_node_services] /node/controller/setupServices params on 127.0.0.1: 9000:services=kv%2Cindex%2Cn1ql&password=asdasd&hostname=127.0.0.1%3A9000&user=Administrator
2016-04-06 08:41:29 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster] settings/web params on 127.0.0.1:9000:username=Administrator&password=asdasd&port=9000
2016-04-06 08:41:29 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster_memoryQuota] pools/default params : memoryQuota=8041
2016-04-06 08:41:29 | INFO | MainProcess | Cluster_Thread | [rest_client.set_indexer_storage_mode] settings/indexes params : storageMode=forestdb
2016-04-06 08:41:29 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster] settings/web params on 127.0.0.1:9001:username=Administrator&password=asdasd&port=9001
2016-04-06 08:41:29 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster_memoryQuota] pools/default params : memoryQuota=8041
2016-04-06 08:41:29 | INFO | MainProcess | Cluster_Thread | [rest_client.set_indexer_storage_mode] settings/indexes params : storageMode=forestdb
2016-04-06 08:41:29 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster] settings/web params on 127.0.0.1:9002:username=Administrator&password=asdasd&port=9002
2016-04-06 08:41:29 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster_memoryQuota] pools/default params : memoryQuota=8041
2016-04-06 08:41:29 | INFO | MainProcess | Cluster_Thread | [rest_client.set_indexer_storage_mode] settings/indexes params : storageMode=forestdb
2016-04-06 08:41:29 | INFO | MainProcess | Cluster_Thread | [rest_client.init_cluster] settings/web params on 127.0.0.1:9003:username=Administrator&password=asdasd&port=9003
2016-04-06 08:41:29 | INFO | MainProcess | test_thread | [basetestcase.setUp] done initializing cluster
2016-04-06 08:41:30 | INFO | MainProcess | Cluster_Thread | [rest_client.create_bucket] http://127.0.0.1:9000/pools/default/buckets with param: bucketType=membase&evictionPolicy=valueOnly&threadsNumber=3&ramQuotaMB=8041&proxyPort=11211&authType=sasl&name=default&flushEnabled=1&replicaNumber=1&replicaIndex=1&saslPassword=None
2016-04-06 08:41:30 | INFO | MainProcess | Cluster_Thread | [rest_client.create_bucket] 0.02 seconds to create bucket default
2016-04-06 08:41:30 | INFO | MainProcess | Cluster_Thread | [bucket_helper.wait_for_memcached] waiting for memcached bucket : default in 127.0.0.1 to accept set ops
2016-04-06 08:41:31 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 127.0.0.1:12000 default
2016-04-06 08:41:31 | INFO | MainProcess | Cluster_Thread | [data_helper.direct_client] creating direct client 127.0.0.1:12000 default
2016-04-06 08:41:31 | INFO | MainProcess | Cluster_Thread | [task.check] bucket 'default' was created with per node RAM quota: 8041
2016-04-06 08:41:31 | INFO | MainProcess | test_thread | [basetestcase.setUp] ==============  basetestcase setup was finished for test #1 test_multi_create_query_explain_drop_index ==============
2016-04-06 08:41:31 | INFO | MainProcess | test_thread | [basetestcase.sleep] sleep for 10 secs.  ...
2016-04-06 08:41:41 | INFO | MainProcess | test_thread | [newtuq.setUp] Initial status of 127.0.0.1 cluster is healthy
2016-04-06 08:41:41 | INFO | MainProcess | test_thread | [newtuq.setUp] current status of 127.0.0.1  is healthy
2016-04-06 08:41:41 | INFO | MainProcess | test_thread | [newtuq.setUp] Initial status of 127.0.0.1 cluster is healthy
2016-04-06 08:41:41 | INFO | MainProcess | test_thread | [newtuq.setUp] current status of 127.0.0.1  is healthy
2016-04-06 08:41:41 | INFO | MainProcess | test_thread | [newtuq.setUp] Initial status of 127.0.0.1 cluster is healthy
2016-04-06 08:41:41 | INFO | MainProcess | test_thread | [newtuq.setUp] current status of 127.0.0.1  is healthy
2016-04-06 08:41:41 | INFO | MainProcess | test_thread | [newtuq.setUp] Initial status of 127.0.0.1 cluster is healthy
2016-04-06 08:41:41 | INFO | MainProcess | test_thread | [newtuq.setUp] current status of 127.0.0.1  is healthy
2016-04-06 08:41:41 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
2016-04-06 08:41:41 | INFO | MainProcess | test_thread | [remote_util.__init__] connecting to 127.0.0.1 with username : Administrator password : asdasd ssh_key: 
2016-04-06 08:41:41 | INFO | MainProcess | test_thread | [remote_util.__init__] Connected to 127.0.0.1
2016-04-06 08:41:42 | INFO | MainProcess | test_thread | [basetestcase.load] create 2016 to default documents...
2016-04-06 08:41:42 | INFO | MainProcess | test_thread | [data_helper.direct_client] creating direct client 127.0.0.1:12000 default
2016-04-06 08:41:44 | INFO | MainProcess | test_thread | [basetestcase.load] LOAD IS FINISHED
2016-04-06 08:41:44 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
2016-04-06 08:41:44 | INFO | MainProcess | test_thread | [newtuq.setUp] ip:127.0.0.1 port:9000 ssh_username:Administrator
number of buckets 1
2016-04-06 08:41:44 | INFO | MainProcess | test_thread | [tuq_helper.create_primary_index] CREATE PRIMARY INDEX ON default  USING GSI
2016-04-06 08:41:44 | INFO | MainProcess | test_thread | [tuq_helper.run_cbq_query] RUN QUERY SELECT * FROM system:indexes where name = '#primary'
2016-04-06 08:41:44 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
2016-04-06 08:41:44 | INFO | MainProcess | test_thread | [tuq_helper.run_cbq_query] TOTAL ELAPSED TIME: 30.885218ms
2016-04-06 08:41:44 | INFO | MainProcess | test_thread | [tuq_helper.run_cbq_query] RUN QUERY CREATE PRIMARY INDEX ON default  USING GSI
2016-04-06 08:41:44 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=CREATE+PRIMARY+INDEX+ON+default++USING+GSI
2016-04-06 08:41:45 | INFO | MainProcess | test_thread | [tuq_helper.run_cbq_query] TOTAL ELAPSED TIME: 500.315281ms
2016-04-06 08:41:45 | INFO | MainProcess | test_thread | [tuq_helper.run_cbq_query] RUN QUERY SELECT * FROM system:indexes where name = '#primary'
2016-04-06 08:41:45 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
2016-04-06 08:41:45 | INFO | MainProcess | test_thread | [tuq_helper.run_cbq_query] TOTAL ELAPSED TIME: 12.211479ms
2016-04-06 08:41:45 | INFO | MainProcess | test_thread | [base_2i.setUp] {'in_between': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}, 'initial': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}, 'after': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}, 'before': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}}
2016-04-06 08:41:45 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
2016-04-06 08:41:45 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
2016-04-06 08:41:45 | INFO | MainProcess | Cluster_Thread | [tuq_helper.run_cbq_query] RUN QUERY CREATE INDEX employeedc0ac4e6aed847288a7fdb8a99c8697bjob_title ON default(job_title) USING GSI  WITH {'nodes': ['127.0.0.1:9000'], 'defer_build': True}
2016-04-06 08:41:45 | INFO | MainProcess | Cluster_Thread | [rest_client.query_tool] query params : statement=CREATE+INDEX+employeedc0ac4e6aed847288a7fdb8a99c8697bjob_title+ON+default%28job_title%29+USING+GSI++WITH+%7B%27nodes%27%3A+%5B%27127.0.0.1%3A9000%27%5D%2C+%27defer_build%27%3A+True%7D
2016-04-06 08:41:45 | INFO | MainProcess | Cluster_Thread | [tuq_helper.run_cbq_query] TOTAL ELAPSED TIME: 31.957059ms
2016-04-06 08:41:45 | INFO | MainProcess | test_thread | [base_2i.async_build_index] BUILD INDEX on default(employeedc0ac4e6aed847288a7fdb8a99c8697bjob_title) USING GSI
2016-04-06 08:41:46 | INFO | MainProcess | Cluster_Thread | [tuq_helper.run_cbq_query] RUN QUERY BUILD INDEX on default(employeedc0ac4e6aed847288a7fdb8a99c8697bjob_title) USING GSI
2016-04-06 08:41:46 | INFO | MainProcess | Cluster_Thread | [rest_client.query_tool] query params : statement=BUILD+INDEX+on+default%28employeedc0ac4e6aed847288a7fdb8a99c8697bjob_title%29+USING+GSI
2016-04-06 08:41:46 | INFO | MainProcess | Cluster_Thread | [tuq_helper.run_cbq_query] TOTAL ELAPSED TIME: 9.226912ms
2016-04-06 08:41:47 | INFO | MainProcess | Cluster_Thread | [tuq_helper.run_cbq_query] RUN QUERY SELECT * FROM system:indexes where name = 'employeedc0ac4e6aed847288a7fdb8a99c8697bjob_title'
2016-04-06 08:41:47 | INFO | MainProcess | Cluster_Thread | [rest_client.query_tool] query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employeedc0ac4e6aed847288a7fdb8a99c8697bjob_title%27
2016-04-06 08:41:47 | INFO | MainProcess | Cluster_Thread | [tuq_helper.run_cbq_query] TOTAL ELAPSED TIME: 15.399854ms
2016-04-06 08:41:48 | INFO | MainProcess | Cluster_Thread | [tuq_helper.run_cbq_query] RUN QUERY SELECT * FROM system:indexes where name = 'employeedc0ac4e6aed847288a7fdb8a99c8697bjob_title'
2016-04-06 08:41:48 | INFO | MainProcess | Cluster_Thread | [rest_client.query_tool] query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employeedc0ac4e6aed847288a7fdb8a99c8697bjob_title%27
2016-04-06 08:41:48 | INFO | MainProcess | Cluster_Thread | [tuq_helper.run_cbq_query] TOTAL ELAPSED TIME: 15.040109ms
2016-04-06 08:41:49 | INFO | MainProcess | Cluster_Thread | [task.execute]  <<<<< START Executing Query EXPLAIN SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
2016-04-06 08:41:49 | INFO | MainProcess | Cluster_Thread | [tuq_helper.run_cbq_query] RUN QUERY EXPLAIN SELECT * FROM default WHERE  job_title = "Sales" 
2016-04-06 08:41:49 | INFO | MainProcess | Cluster_Thread | [rest_client.query_tool] query params : statement=EXPLAIN+SELECT+%2A+FROM+default+WHERE++job_title+%3D+%22Sales%22+
2016-04-06 08:41:49 | INFO | MainProcess | Cluster_Thread | [tuq_helper.run_cbq_query] TOTAL ELAPSED TIME: 1.89247ms
2016-04-06 08:41:49 | INFO | MainProcess | Cluster_Thread | [task.execute] {u'status': u'success', u'metrics': {u'elapsedTime': u'1.89247ms', u'executionTime': u'1.825876ms', u'resultSize': 2409, u'resultCount': 1}, u'results': [{u'text': u'SELECT * FROM default WHERE  job_title = "Sales"', u'plan': {u'#operator': u'Sequence', u'~children': [{u'index': u'employeedc0ac4e6aed847288a7fdb8a99c8697bjob_title', u'namespace': u'default', u'keyspace': u'default', u'#operator': u'IndexScan', u'spans': [{u'Range': {u'High': [u'"Sales"'], u'Low': [u'"Sales"'], u'Inclusion': 3}}], u'using': u'gsi', u'index_id': u'e94795b8bf35f4d6'}, {u'#operator': u'Parallel', u'~child': {u'#operator': u'Sequence', u'~children': [{u'keyspace': u'default', u'#operator': u'Fetch', u'namespace': u'default'}, {u'#operator': u'Filter', u'condition': u'((`default`.`job_title`) = "Sales")'}, {u'#operator': u'InitialProject', u'result_terms': [{u'expr': u'self', u'star': True}]}, {u'#operator': u'FinalProject'}]}}]}}], u'requestID': u'3f2472a0-32cf-443a-95bc-9806d7b11a96', u'signature': u'json'}
2016-04-06 08:41:49 | INFO | MainProcess | Cluster_Thread | [task.execute]  <<<<< Done Executing Query EXPLAIN SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
2016-04-06 08:41:49 | INFO | MainProcess | Cluster_Thread | [task.check]  <<<<< Done VERIFYING Query EXPLAIN SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
2016-04-06 08:41:49 | INFO | MainProcess | test_thread | [base_2i.async_query_using_index] Query : SELECT * FROM default WHERE  job_title = "Sales" 
2016-04-06 08:41:49 | INFO | MainProcess | test_thread | [tuq_generators.generate_expected_result] FROM clause ===== is default
2016-04-06 08:41:49 | INFO | MainProcess | test_thread | [tuq_generators.generate_expected_result] WHERE clause ===== is   doc["job_title"] == "Sales" 
2016-04-06 08:41:49 | INFO | MainProcess | test_thread | [tuq_generators.generate_expected_result] UNNEST clause ===== is None
2016-04-06 08:41:49 | INFO | MainProcess | test_thread | [tuq_generators.generate_expected_result] SELECT clause ===== is {"*" : doc,}
2016-04-06 08:41:50 | INFO | MainProcess | Cluster_Thread | [task.execute]  <<<<< START Executing Query SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
2016-04-06 08:41:50 | INFO | MainProcess | Cluster_Thread | [tuq_helper.run_cbq_query] RUN QUERY SELECT * FROM default WHERE  job_title = "Sales" 
2016-04-06 08:41:50 | INFO | MainProcess | Cluster_Thread | [rest_client.query_tool] query params : scan_consistency=request_plus&statement=SELECT+%2A+FROM+default+WHERE++job_title+%3D+%22Sales%22+
2016-04-06 08:41:50 | INFO | MainProcess | Cluster_Thread | [tuq_helper.run_cbq_query] TOTAL ELAPSED TIME: 163.702127ms
2016-04-06 08:41:50 | INFO | MainProcess | Cluster_Thread | [tuq_helper._verify_results]  Analyzing Actual Result
2016-04-06 08:41:50 | INFO | MainProcess | Cluster_Thread | [tuq_helper._verify_results]  Analyzing Expected Result
2016-04-06 08:41:50 | INFO | MainProcess | Cluster_Thread | [task.execute]  <<<<< Done Executing Query SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
2016-04-06 08:41:50 | INFO | MainProcess | Cluster_Thread | [task.check]  <<<<< Done VERIFYING Query SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
2016-04-06 08:41:51 | INFO | MainProcess | Cluster_Thread | [tuq_helper.run_cbq_query] RUN QUERY SELECT * FROM system:indexes where name = 'employeedc0ac4e6aed847288a7fdb8a99c8697bjob_title'
2016-04-06 08:41:51 | INFO | MainProcess | Cluster_Thread | [rest_client.query_tool] query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employeedc0ac4e6aed847288a7fdb8a99c8697bjob_title%27
2016-04-06 08:41:51 | INFO | MainProcess | Cluster_Thread | [tuq_helper.run_cbq_query] TOTAL ELAPSED TIME: 15.555162ms
2016-04-06 08:41:51 | INFO | MainProcess | Cluster_Thread | [tuq_helper.run_cbq_query] RUN QUERY DROP INDEX default.employeedc0ac4e6aed847288a7fdb8a99c8697bjob_title USING GSI
2016-04-06 08:41:51 | INFO | MainProcess | Cluster_Thread | [rest_client.query_tool] query params : statement=DROP+INDEX+default.employeedc0ac4e6aed847288a7fdb8a99c8697bjob_title+USING+GSI
2016-04-06 08:41:51 | INFO | MainProcess | Cluster_Thread | [tuq_helper.run_cbq_query] TOTAL ELAPSED TIME: 11.648307ms
2016-04-06 08:41:51 | INFO | MainProcess | Cluster_Thread | [tuq_helper.run_cbq_query] RUN QUERY SELECT * FROM system:indexes where name = 'employeedc0ac4e6aed847288a7fdb8a99c8697bjob_title'
2016-04-06 08:41:51 | INFO | MainProcess | Cluster_Thread | [rest_client.query_tool] query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employeedc0ac4e6aed847288a7fdb8a99c8697bjob_title%27
2016-04-06 08:41:51 | INFO | MainProcess | Cluster_Thread | [tuq_helper.run_cbq_query] TOTAL ELAPSED TIME: 13.256042ms
2016-04-06 08:41:51 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
2016-04-06 08:41:51 | INFO | MainProcess | test_thread | [remote_util.__init__] connecting to 127.0.0.1 with username : Administrator password : asdasd ssh_key: 
2016-04-06 08:41:51 | INFO | MainProcess | test_thread | [remote_util.__init__] Connected to 127.0.0.1
2016-04-06 08:41:51 | INFO | MainProcess | test_thread | [rest_client.diag_eval] /diag/eval status on 127.0.0.1:9000: True content: "/opt/build/ns_server/logs/n_0" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))).
2016-04-06 08:41:52 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 127.0.0.1: zgrep "panic" "/opt/build/ns_server/logs/n_0"/indexer.log* | wc -l
2016-04-06 08:41:52 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully
2016-04-06 08:41:52 | INFO | MainProcess | test_thread | [basetestcase.get_nodes_from_services_map] list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
2016-04-06 08:41:52 | INFO | MainProcess | test_thread | [tuq_helper.drop_primary_index] CHECK FOR PRIMARY INDEXES
2016-04-06 08:41:52 | INFO | MainProcess | test_thread | [tuq_helper.drop_primary_index] DROP PRIMARY INDEX ON default USING GSI
2016-04-06 08:41:52 | INFO | MainProcess | test_thread | [tuq_helper.run_cbq_query] RUN QUERY SELECT * FROM system:indexes where name = '#primary'
2016-04-06 08:41:52 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
2016-04-06 08:41:52 | INFO | MainProcess | test_thread | [tuq_helper.run_cbq_query] TOTAL ELAPSED TIME: 12.07373ms
2016-04-06 08:41:52 | INFO | MainProcess | test_thread | [tuq_helper.run_cbq_query] RUN QUERY DROP PRIMARY INDEX ON default USING GSI
2016-04-06 08:41:52 | INFO | MainProcess | test_thread | [rest_client.query_tool] query params : statement=DROP+PRIMARY+INDEX+ON+default+USING+GSI
2016-04-06 08:41:52 | INFO | MainProcess | test_thread | [tuq_helper.run_cbq_query] TOTAL ELAPSED TIME: 12.330224ms
2016-04-06 08:41:52 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 127.0.0.1: rm -f /opt/couchbase/var/lib/couchbase/meta;rm -f /tmp/log_upr_client.sock
2016-04-06 08:41:52 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully
2016-04-06 08:41:52 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 127.0.0.1: killall cbq-engine
2016-04-06 08:41:52 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully
2016-04-06 08:41:52 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 127.0.0.1: killall tuqtng
2016-04-06 08:41:52 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully
2016-04-06 08:41:52 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on 127.0.0.1: killall indexer
2016-04-06 08:41:52 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully
2016-04-06 08:41:52 | INFO | MainProcess | test_thread | [basetestcase.tearDown] ==============  basetestcase cleanup was started for test #1 test_multi_create_query_explain_drop_index ==============
2016-04-06 08:41:52 | INFO | MainProcess | test_thread | [bucket_helper.delete_all_buckets_or_assert] deleting existing buckets [u'default'] on 127.0.0.1
2016-04-06 08:41:52 | INFO | MainProcess | test_thread | [bucket_helper.delete_all_buckets_or_assert] remove bucket default ...
2016-04-06 08:41:54 | INFO | MainProcess | test_thread | [bucket_helper.delete_all_buckets_or_assert] deleted bucket : default from 127.0.0.1
2016-04-06 08:41:54 | INFO | MainProcess | test_thread | [bucket_helper.wait_for_bucket_deletion] waiting for bucket deletion to complete....
2016-04-06 08:41:54 | INFO | MainProcess | test_thread | [rest_client.bucket_exists] node 127.0.0.1 existing buckets : []
2016-04-06 08:41:54 | INFO | MainProcess | test_thread | [bucket_helper.delete_all_buckets_or_assert] deleting existing buckets [] on 127.0.0.1
2016-04-06 08:41:54 | INFO | MainProcess | test_thread | [bucket_helper.delete_all_buckets_or_assert] deleting existing buckets [] on 127.0.0.1
2016-04-06 08:41:54 | INFO | MainProcess | test_thread | [bucket_helper.delete_all_buckets_or_assert] deleting existing buckets [] on 127.0.0.1
2016-04-06 08:41:54 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] waiting for ns_server @ 127.0.0.1:9000
2016-04-06 08:41:54 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] ns_server @ 127.0.0.1:9000 is running
2016-04-06 08:41:54 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] waiting for ns_server @ 127.0.0.1:9001
2016-04-06 08:41:54 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] ns_server @ 127.0.0.1:9001 is running
2016-04-06 08:41:54 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] waiting for ns_server @ 127.0.0.1:9002
2016-04-06 08:41:54 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] ns_server @ 127.0.0.1:9002 is running
2016-04-06 08:41:54 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] waiting for ns_server @ 127.0.0.1:9003
2016-04-06 08:41:54 | INFO | MainProcess | test_thread | [cluster_helper.wait_for_ns_servers_or_assert] ns_server @ 127.0.0.1:9003 is running
2016-04-06 08:41:54 | INFO | MainProcess | test_thread | [basetestcase.tearDown] ==============  basetestcase cleanup was finished for test #1 test_multi_create_query_explain_drop_index ==============
ok

----------------------------------------------------------------------
Ran 1 test in 46.492s

OK
test_multi_create_query_explain_drop_index (2i.indexscans_2i.SecondaryIndexingScanTests) ... Cluster instance shutdown with force
summary so far suite 2i.indexscans_2i.SecondaryIndexingScanTests , pass 1 , fail 0
testrunner logs, diags and results are available under /opt/build/testrunner/logs/testrunner-16-Apr-06_08-41-08/test_1
Logs will be stored at /opt/build/testrunner/logs/testrunner-16-Apr-06_08-41-08/test_2

./testrunner -i b/resources/dev-4-nodes-xdcr_n1ql_2i.ini makefile=True -t 2i.indexscans_2i.SecondaryIndexingScanTests.test_multi_create_query_explain_drop_index,groups=simple:equals:no_orderby_groupby:range,dataset=default,doc-per-day=1,use_gsi_for_primary=True,use_gsi_for_secondary=False

Test Input params:
{'doc-per-day': '1', 'use_gsi_for_primary': 'True', 'use_gsi_for_secondary': 'False', 'makefile': 'True', 'conf_file': 'conf/simple_2i_n1ql.conf', 'dataset': 'default', 'cluster_name': 'dev-4-nodes-xdcr_n1ql_2i', 'ini': 'b/resources/dev-4-nodes-xdcr_n1ql_2i.ini', 'groups': 'simple:equals:no_orderby_groupby:range', 'case_number': 2, 'num_nodes': 4, 'logs_folder': '/opt/build/testrunner/logs/testrunner-16-Apr-06_08-41-08/test_2', 'spec': 'simple_2i_n1ql'}
[2016-04-06 08:41:54,699] - [rest_client:1636] INFO - Node versions in cluster [u'4.5.35007-1459908767-rel-enterprise']
[2016-04-06 08:41:54,699] - [basetestcase:146] INFO - ==============  basetestcase setup was started for test #2 test_multi_create_query_explain_drop_index==============
[2016-04-06 08:41:54,710] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:41:54,715] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:41:54,720] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:41:54,724] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:41:54,734] - [basetestcase:365] INFO - sleep for 10 secs.  ...
[2016-04-06 08:42:04,745] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9000
[2016-04-06 08:42:04,748] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9000 is running
[2016-04-06 08:42:04,751] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9001
[2016-04-06 08:42:04,753] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9001 is running
[2016-04-06 08:42:04,756] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9002
[2016-04-06 08:42:04,761] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9002 is running
[2016-04-06 08:42:04,763] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9003
[2016-04-06 08:42:04,766] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9003 is running
[2016-04-06 08:42:04,766] - [basetestcase:167] INFO - initializing cluster
[2016-04-06 08:42:05,710] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:42:05,713] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:42:05,717] - [rest_client:789] INFO - /node/controller/setupServices params on 127.0.0.1: 9000:services=kv%2Cindex%2Cn1ql&password=asdasd&hostname=127.0.0.1%3A9000&user=Administrator
[2016-04-06 08:42:05,732] - [rest_client:757] ERROR - http://127.0.0.1:9000//node/controller/setupServices error 400 reason: unknown ["cannot change node services after cluster is provisioned"]
[2016-04-06 08:42:05,732] - [rest_client:794] INFO - This node is already provisioned with services, we do not consider this as failure for test case
[2016-04-06 08:42:05,733] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9000:username=Administrator&password=asdasd&port=9000
[2016-04-06 08:42:05,753] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:42:05,758] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:42:05,761] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9001:username=Administrator&password=asdasd&port=9001
[2016-04-06 08:42:05,780] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:42:05,784] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:42:05,787] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9002:username=Administrator&password=asdasd&port=9002
[2016-04-06 08:42:05,809] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:42:05,812] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:42:05,816] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9003:username=Administrator&password=asdasd&port=9003
[2016-04-06 08:42:05,826] - [basetestcase:187] INFO - done initializing cluster
[2016-04-06 08:42:06,835] - [rest_client:1772] INFO - http://127.0.0.1:9000/pools/default/buckets with param: bucketType=membase&evictionPolicy=valueOnly&threadsNumber=3&ramQuotaMB=8041&proxyPort=11211&authType=sasl&name=default&flushEnabled=1&replicaNumber=1&replicaIndex=1&saslPassword=None
[2016-04-06 08:42:06,839] - [rest_client:1794] INFO - 0.00 seconds to create bucket default
[2016-04-06 08:42:06,840] - [bucket_helper:305] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2016-04-06 08:42:07,907] - [data_helper:295] INFO - creating direct client 127.0.0.1:12000 default
[2016-04-06 08:42:07,946] - [data_helper:295] INFO - creating direct client 127.0.0.1:12000 default
[2016-04-06 08:42:07,970] - [task:288] INFO - bucket 'default' was created with per node RAM quota: 8041
[2016-04-06 08:42:08,001] - [basetestcase:257] INFO - ==============  basetestcase setup was finished for test #2 test_multi_create_query_explain_drop_index ==============
[2016-04-06 08:42:08,008] - [basetestcase:365] INFO - sleep for 10 secs.  ...
[2016-04-06 08:42:18,016] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:42:18,016] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:42:18,022] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:42:18,022] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:42:18,028] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:42:18,028] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:42:18,034] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:42:18,034] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:42:18,040] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:42:18,040] - [remote_util:168] INFO - connecting to 127.0.0.1 with username : Administrator password : asdasd ssh_key: 
[2016-04-06 08:42:18,040] - [remote_util:201] INFO - Connected to 127.0.0.1
[2016-04-06 08:42:18,979] - [basetestcase:2084] INFO - create 2016 to default documents...
[2016-04-06 08:42:18,999] - [data_helper:295] INFO - creating direct client 127.0.0.1:12000 default
[2016-04-06 08:42:21,135] - [basetestcase:2095] INFO - LOAD IS FINISHED
[2016-04-06 08:42:21,147] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:42:21,147] - [newtuq:92] INFO - ip:127.0.0.1 port:9000 ssh_username:Administrator
number of buckets 1
[2016-04-06 08:42:21,147] - [tuq_helper:419] INFO - CREATE PRIMARY INDEX ON default  USING GSI
[2016-04-06 08:42:21,147] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2016-04-06 08:42:21,150] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2016-04-06 08:42:21,185] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 33.767392ms
[2016-04-06 08:42:21,186] - [tuq_helper:75] INFO - RUN QUERY CREATE PRIMARY INDEX ON default  USING GSI
[2016-04-06 08:42:21,189] - [rest_client:2508] INFO - query params : statement=CREATE+PRIMARY+INDEX+ON+default++USING+GSI
[2016-04-06 08:42:21,688] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 497.456647ms
[2016-04-06 08:42:21,688] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2016-04-06 08:42:21,691] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2016-04-06 08:42:21,707] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 13.88991ms
[2016-04-06 08:42:21,708] - [base_2i:48] INFO - {'in_between': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}, 'initial': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}, 'after': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}, 'before': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}}
[2016-04-06 08:42:21,719] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:42:21,728] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:42:21,990] - [tuq_helper:75] INFO - RUN QUERY CREATE INDEX employee8ba64ea70c4f4e59aab0990b8b2f02fbjob_title ON default(job_title) USING VIEW 
[2016-04-06 08:42:21,993] - [rest_client:2508] INFO - query params : statement=CREATE+INDEX+employee8ba64ea70c4f4e59aab0990b8b2f02fbjob_title+ON+default%28job_title%29+USING+VIEW+
[2016-04-06 08:42:22,358] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 363.314006ms
[2016-04-06 08:42:22,359] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee8ba64ea70c4f4e59aab0990b8b2f02fbjob_title'
[2016-04-06 08:42:22,362] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee8ba64ea70c4f4e59aab0990b8b2f02fbjob_title%27
[2016-04-06 08:42:22,385] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 21.41079ms
[2016-04-06 08:42:23,387] - [task:2250] INFO -  <<<<< START Executing Query EXPLAIN SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2016-04-06 08:42:23,387] - [tuq_helper:75] INFO - RUN QUERY EXPLAIN SELECT * FROM default WHERE  job_title = "Sales" 
[2016-04-06 08:42:23,391] - [rest_client:2508] INFO - query params : statement=EXPLAIN+SELECT+%2A+FROM+default+WHERE++job_title+%3D+%22Sales%22+
[2016-04-06 08:42:23,394] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 1.611215ms
[2016-04-06 08:42:23,394] - [task:2259] INFO - {u'status': u'success', u'metrics': {u'elapsedTime': u'1.611215ms', u'executionTime': u'1.544978ms', u'resultSize': 2443, u'resultCount': 1}, u'results': [{u'text': u'SELECT * FROM default WHERE  job_title = "Sales"', u'plan': {u'#operator': u'Sequence', u'~children': [{u'index': u'employee8ba64ea70c4f4e59aab0990b8b2f02fbjob_title', u'namespace': u'default', u'keyspace': u'default', u'#operator': u'IndexScan', u'spans': [{u'Range': {u'High': [u'"Sales"'], u'Low': [u'"Sales"'], u'Inclusion': 3}}], u'using': u'view', u'index_id': u'employee8ba64ea70c4f4e59aab0990b8b2f02fbjob_title'}, {u'#operator': u'Parallel', u'~child': {u'#operator': u'Sequence', u'~children': [{u'keyspace': u'default', u'#operator': u'Fetch', u'namespace': u'default'}, {u'#operator': u'Filter', u'condition': u'((`default`.`job_title`) = "Sales")'}, {u'#operator': u'InitialProject', u'result_terms': [{u'expr': u'self', u'star': True}]}, {u'#operator': u'FinalProject'}]}}]}}], u'requestID': u'5ed664bc-56c9-4634-af0a-ce370a03b6ac', u'signature': u'json'}
[2016-04-06 08:42:23,394] - [task:2260] INFO -  <<<<< Done Executing Query EXPLAIN SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2016-04-06 08:42:23,394] - [task:2289] INFO -  <<<<< Done VERIFYING Query EXPLAIN SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2016-04-06 08:42:23,395] - [base_2i:294] INFO - Query : SELECT * FROM default WHERE  job_title = "Sales" 
[2016-04-06 08:42:23,395] - [tuq_generators:66] INFO - FROM clause ===== is default
[2016-04-06 08:42:23,395] - [tuq_generators:68] INFO - WHERE clause ===== is   doc["job_title"] == "Sales" 
[2016-04-06 08:42:23,395] - [tuq_generators:70] INFO - UNNEST clause ===== is None
[2016-04-06 08:42:23,396] - [tuq_generators:72] INFO - SELECT clause ===== is {"*" : doc,}
[2016-04-06 08:42:24,396] - [task:2250] INFO -  <<<<< START Executing Query SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2016-04-06 08:42:24,396] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM default WHERE  job_title = "Sales" 
[2016-04-06 08:42:24,399] - [rest_client:2508] INFO - query params : scan_consistency=request_plus&statement=SELECT+%2A+FROM+default+WHERE++job_title+%3D+%22Sales%22+
[2016-04-06 08:42:24,603] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 189.213852ms
[2016-04-06 08:42:24,621] - [tuq_helper:108] INFO -  Analyzing Actual Result
[2016-04-06 08:42:24,621] - [tuq_helper:110] INFO -  Analyzing Expected Result
[2016-04-06 08:42:24,624] - [task:2260] INFO -  <<<<< Done Executing Query SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2016-04-06 08:42:24,624] - [task:2289] INFO -  <<<<< Done VERIFYING Query SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2016-04-06 08:42:25,625] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee8ba64ea70c4f4e59aab0990b8b2f02fbjob_title'
[2016-04-06 08:42:25,629] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee8ba64ea70c4f4e59aab0990b8b2f02fbjob_title%27
[2016-04-06 08:42:25,650] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 18.737736ms
[2016-04-06 08:42:25,650] - [tuq_helper:75] INFO - RUN QUERY DROP INDEX default.employee8ba64ea70c4f4e59aab0990b8b2f02fbjob_title USING VIEW 
[2016-04-06 08:42:25,653] - [rest_client:2508] INFO - query params : statement=DROP+INDEX+default.employee8ba64ea70c4f4e59aab0990b8b2f02fbjob_title+USING+VIEW+
[2016-04-06 08:42:25,664] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 9.836548ms
[2016-04-06 08:42:25,664] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee8ba64ea70c4f4e59aab0990b8b2f02fbjob_title'
[2016-04-06 08:42:25,674] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee8ba64ea70c4f4e59aab0990b8b2f02fbjob_title%27
[2016-04-06 08:42:25,691] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 16.122746ms
[2016-04-06 08:42:25,697] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:42:25,697] - [remote_util:168] INFO - connecting to 127.0.0.1 with username : Administrator password : asdasd ssh_key: 
[2016-04-06 08:42:25,697] - [remote_util:201] INFO - Connected to 127.0.0.1
[2016-04-06 08:42:25,707] - [rest_client:1222] INFO - /diag/eval status on 127.0.0.1:9000: True content: "/opt/build/ns_server/logs/n_0" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))).
[2016-04-06 08:42:25,856] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: zgrep "panic" "/opt/build/ns_server/logs/n_0"/indexer.log* | wc -l
[2016-04-06 08:42:25,879] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:42:25,886] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:42:25,886] - [tuq_helper:392] INFO - CHECK FOR PRIMARY INDEXES
[2016-04-06 08:42:25,886] - [tuq_helper:399] INFO - DROP PRIMARY INDEX ON default USING GSI
[2016-04-06 08:42:25,886] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2016-04-06 08:42:25,889] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2016-04-06 08:42:25,909] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 18.108365ms
[2016-04-06 08:42:25,909] - [tuq_helper:75] INFO - RUN QUERY DROP PRIMARY INDEX ON default USING GSI
[2016-04-06 08:42:25,912] - [rest_client:2508] INFO - query params : statement=DROP+PRIMARY+INDEX+ON+default+USING+GSI
[2016-04-06 08:42:25,937] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 22.57476ms
[2016-04-06 08:42:26,083] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: rm -f /opt/couchbase/var/lib/couchbase/meta;rm -f /tmp/log_upr_client.sock
[2016-04-06 08:42:26,103] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:42:26,104] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: killall cbq-engine
[2016-04-06 08:42:26,129] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:42:26,129] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: killall tuqtng
[2016-04-06 08:42:26,165] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:42:26,165] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: killall indexer
[2016-04-06 08:42:26,197] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:42:26,198] - [basetestcase:297] INFO - ==============  basetestcase cleanup was started for test #2 test_multi_create_query_explain_drop_index ==============
[2016-04-06 08:42:26,220] - [bucket_helper:137] INFO - deleting existing buckets [u'default'] on 127.0.0.1
[2016-04-06 08:42:26,220] - [bucket_helper:139] INFO - remove bucket default ...
[2016-04-06 08:42:27,039] - [bucket_helper:153] INFO - deleted bucket : default from 127.0.0.1
[2016-04-06 08:42:27,040] - [bucket_helper:226] INFO - waiting for bucket deletion to complete....
[2016-04-06 08:42:27,042] - [rest_client:127] INFO - node 127.0.0.1 existing buckets : []
[2016-04-06 08:42:27,048] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:42:27,054] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:42:27,059] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:42:27,075] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9000
[2016-04-06 08:42:27,078] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9000 is running
[2016-04-06 08:42:27,080] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9001
[2016-04-06 08:42:27,083] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9001 is running
[2016-04-06 08:42:27,086] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9002
[2016-04-06 08:42:27,089] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9002 is running
[2016-04-06 08:42:27,092] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9003
[2016-04-06 08:42:27,094] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9003 is running
[2016-04-06 08:42:27,095] - [basetestcase:315] INFO - ==============  basetestcase cleanup was finished for test #2 test_multi_create_query_explain_drop_index ==============
ok

----------------------------------------------------------------------
Ran 1 test in 32.409s

OK
test_multi_create_query_explain_drop_index (2i.indexscans_2i.SecondaryIndexingScanTests) ... Cluster instance shutdown with force
summary so far suite 2i.indexscans_2i.SecondaryIndexingScanTests , pass 2 , fail 0
testrunner logs, diags and results are available under /opt/build/testrunner/logs/testrunner-16-Apr-06_08-41-08/test_2
Logs will be stored at /opt/build/testrunner/logs/testrunner-16-Apr-06_08-41-08/test_3

./testrunner -i b/resources/dev-4-nodes-xdcr_n1ql_2i.ini makefile=True -t 2i.indexscans_2i.SecondaryIndexingScanTests.test_multi_create_query_explain_drop_index,groups=simple:equals:no_orderby_groupby:range,dataset=default,doc-per-day=1,use_gsi_for_primary=False,use_gsi_for_secondary=True

Test Input params:
{'doc-per-day': '1', 'use_gsi_for_primary': 'False', 'use_gsi_for_secondary': 'True', 'makefile': 'True', 'conf_file': 'conf/simple_2i_n1ql.conf', 'dataset': 'default', 'cluster_name': 'dev-4-nodes-xdcr_n1ql_2i', 'ini': 'b/resources/dev-4-nodes-xdcr_n1ql_2i.ini', 'groups': 'simple:equals:no_orderby_groupby:range', 'case_number': 3, 'num_nodes': 4, 'logs_folder': '/opt/build/testrunner/logs/testrunner-16-Apr-06_08-41-08/test_3', 'spec': 'simple_2i_n1ql'}
[2016-04-06 08:42:27,122] - [rest_client:1636] INFO - Node versions in cluster [u'4.5.35007-1459908767-rel-enterprise']
[2016-04-06 08:42:27,122] - [basetestcase:146] INFO - ==============  basetestcase setup was started for test #3 test_multi_create_query_explain_drop_index==============
[2016-04-06 08:42:27,133] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:42:27,138] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:42:27,142] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:42:27,147] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:42:27,157] - [basetestcase:365] INFO - sleep for 10 secs.  ...
[2016-04-06 08:42:37,170] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9000
[2016-04-06 08:42:37,173] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9000 is running
[2016-04-06 08:42:37,176] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9001
[2016-04-06 08:42:37,179] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9001 is running
[2016-04-06 08:42:37,182] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9002
[2016-04-06 08:42:37,185] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9002 is running
[2016-04-06 08:42:37,189] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9003
[2016-04-06 08:42:37,191] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9003 is running
[2016-04-06 08:42:37,191] - [basetestcase:167] INFO - initializing cluster
[2016-04-06 08:42:38,136] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:42:38,139] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:42:38,143] - [rest_client:789] INFO - /node/controller/setupServices params on 127.0.0.1: 9000:services=kv%2Cindex%2Cn1ql&password=asdasd&hostname=127.0.0.1%3A9000&user=Administrator
[2016-04-06 08:42:38,162] - [rest_client:757] ERROR - http://127.0.0.1:9000//node/controller/setupServices error 400 reason: unknown ["cannot change node services after cluster is provisioned"]
[2016-04-06 08:42:38,163] - [rest_client:794] INFO - This node is already provisioned with services, we do not consider this as failure for test case
[2016-04-06 08:42:38,163] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9000:username=Administrator&password=asdasd&port=9000
[2016-04-06 08:42:38,179] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:42:38,184] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:42:38,189] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9001:username=Administrator&password=asdasd&port=9001
[2016-04-06 08:42:38,205] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:42:38,208] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:42:38,212] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9002:username=Administrator&password=asdasd&port=9002
[2016-04-06 08:42:38,228] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:42:38,231] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:42:38,236] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9003:username=Administrator&password=asdasd&port=9003
[2016-04-06 08:42:38,246] - [basetestcase:187] INFO - done initializing cluster
[2016-04-06 08:42:39,255] - [rest_client:1772] INFO - http://127.0.0.1:9000/pools/default/buckets with param: bucketType=membase&evictionPolicy=valueOnly&threadsNumber=3&ramQuotaMB=8041&proxyPort=11211&authType=sasl&name=default&flushEnabled=1&replicaNumber=1&replicaIndex=1&saslPassword=None
[2016-04-06 08:42:39,261] - [rest_client:1794] INFO - 0.01 seconds to create bucket default
[2016-04-06 08:42:39,261] - [bucket_helper:305] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2016-04-06 08:42:40,327] - [data_helper:295] INFO - creating direct client 127.0.0.1:12000 default
[2016-04-06 08:42:40,367] - [data_helper:295] INFO - creating direct client 127.0.0.1:12000 default
[2016-04-06 08:42:40,397] - [task:288] INFO - bucket 'default' was created with per node RAM quota: 8041
[2016-04-06 08:42:40,421] - [basetestcase:257] INFO - ==============  basetestcase setup was finished for test #3 test_multi_create_query_explain_drop_index ==============
[2016-04-06 08:42:40,426] - [basetestcase:365] INFO - sleep for 10 secs.  ...
[2016-04-06 08:42:50,436] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:42:50,437] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:42:50,443] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:42:50,443] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:42:50,448] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:42:50,449] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:42:50,454] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:42:50,454] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:42:50,459] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:42:50,460] - [remote_util:168] INFO - connecting to 127.0.0.1 with username : Administrator password : asdasd ssh_key: 
[2016-04-06 08:42:50,460] - [remote_util:201] INFO - Connected to 127.0.0.1
[2016-04-06 08:42:51,401] - [basetestcase:2084] INFO - create 2016 to default documents...
[2016-04-06 08:42:51,421] - [data_helper:295] INFO - creating direct client 127.0.0.1:12000 default
[2016-04-06 08:42:53,589] - [basetestcase:2095] INFO - LOAD IS FINISHED
[2016-04-06 08:42:53,599] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:42:53,599] - [newtuq:92] INFO - ip:127.0.0.1 port:9000 ssh_username:Administrator
number of buckets 1
[2016-04-06 08:42:53,600] - [tuq_helper:419] INFO - CREATE PRIMARY INDEX ON default  USING VIEW 
[2016-04-06 08:42:53,600] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2016-04-06 08:42:53,602] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2016-04-06 08:42:53,634] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 29.487306ms
[2016-04-06 08:42:53,634] - [tuq_helper:75] INFO - RUN QUERY CREATE PRIMARY INDEX ON default  USING VIEW 
[2016-04-06 08:42:53,637] - [rest_client:2508] INFO - query params : statement=CREATE+PRIMARY+INDEX+ON+default++USING+VIEW+
[2016-04-06 08:42:54,008] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 368.010599ms
[2016-04-06 08:42:54,008] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2016-04-06 08:42:54,011] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2016-04-06 08:42:54,030] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 17.453503ms
[2016-04-06 08:42:54,031] - [base_2i:48] INFO - {'in_between': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}, 'initial': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}, 'after': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}, 'before': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}}
[2016-04-06 08:42:54,042] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:42:54,047] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:42:54,415] - [tuq_helper:75] INFO - RUN QUERY CREATE INDEX employee5f47e79e4eb942fa9cf6ef6a51526140job_title ON default(job_title) USING GSI  WITH {'nodes': ['127.0.0.1:9000'], 'defer_build': True}
[2016-04-06 08:42:54,419] - [rest_client:2508] INFO - query params : statement=CREATE+INDEX+employee5f47e79e4eb942fa9cf6ef6a51526140job_title+ON+default%28job_title%29+USING+GSI++WITH+%7B%27nodes%27%3A+%5B%27127.0.0.1%3A9000%27%5D%2C+%27defer_build%27%3A+True%7D
[2016-04-06 08:42:54,470] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 49.605991ms
[2016-04-06 08:42:54,471] - [base_2i:102] INFO - BUILD INDEX on default(employee5f47e79e4eb942fa9cf6ef6a51526140job_title) USING GSI
[2016-04-06 08:42:55,472] - [tuq_helper:75] INFO - RUN QUERY BUILD INDEX on default(employee5f47e79e4eb942fa9cf6ef6a51526140job_title) USING GSI
[2016-04-06 08:42:55,475] - [rest_client:2508] INFO - query params : statement=BUILD+INDEX+on+default%28employee5f47e79e4eb942fa9cf6ef6a51526140job_title%29+USING+GSI
[2016-04-06 08:42:55,501] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 24.337162ms
[2016-04-06 08:42:56,502] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee5f47e79e4eb942fa9cf6ef6a51526140job_title'
[2016-04-06 08:42:56,505] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee5f47e79e4eb942fa9cf6ef6a51526140job_title%27
[2016-04-06 08:42:56,524] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 17.131559ms
[2016-04-06 08:42:57,525] - [task:2250] INFO -  <<<<< START Executing Query EXPLAIN SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2016-04-06 08:42:57,526] - [tuq_helper:75] INFO - RUN QUERY EXPLAIN SELECT * FROM default WHERE  job_title = "Sales" 
[2016-04-06 08:42:57,529] - [rest_client:2508] INFO - query params : statement=EXPLAIN+SELECT+%2A+FROM+default+WHERE++job_title+%3D+%22Sales%22+
[2016-04-06 08:42:57,533] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 3.390526ms
[2016-04-06 08:42:57,534] - [task:2259] INFO - {u'status': u'success', u'metrics': {u'elapsedTime': u'3.390526ms', u'executionTime': u'3.336339ms', u'resultSize': 2409, u'resultCount': 1}, u'results': [{u'text': u'SELECT * FROM default WHERE  job_title = "Sales"', u'plan': {u'#operator': u'Sequence', u'~children': [{u'index': u'employee5f47e79e4eb942fa9cf6ef6a51526140job_title', u'namespace': u'default', u'keyspace': u'default', u'#operator': u'IndexScan', u'spans': [{u'Range': {u'High': [u'"Sales"'], u'Low': [u'"Sales"'], u'Inclusion': 3}}], u'using': u'gsi', u'index_id': u'd4c0ace68d054be0'}, {u'#operator': u'Parallel', u'~child': {u'#operator': u'Sequence', u'~children': [{u'keyspace': u'default', u'#operator': u'Fetch', u'namespace': u'default'}, {u'#operator': u'Filter', u'condition': u'((`default`.`job_title`) = "Sales")'}, {u'#operator': u'InitialProject', u'result_terms': [{u'expr': u'self', u'star': True}]}, {u'#operator': u'FinalProject'}]}}]}}], u'requestID': u'b2a8af60-6b3c-451d-92c8-b06579b15549', u'signature': u'json'}
[2016-04-06 08:42:57,534] - [task:2260] INFO -  <<<<< Done Executing Query EXPLAIN SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2016-04-06 08:42:57,534] - [task:2289] INFO -  <<<<< Done VERIFYING Query EXPLAIN SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2016-04-06 08:42:57,534] - [base_2i:294] INFO - Query : SELECT * FROM default WHERE  job_title = "Sales" 
[2016-04-06 08:42:57,534] - [tuq_generators:66] INFO - FROM clause ===== is default
[2016-04-06 08:42:57,535] - [tuq_generators:68] INFO - WHERE clause ===== is   doc["job_title"] == "Sales" 
[2016-04-06 08:42:57,535] - [tuq_generators:70] INFO - UNNEST clause ===== is None
[2016-04-06 08:42:57,535] - [tuq_generators:72] INFO - SELECT clause ===== is {"*" : doc,}
[2016-04-06 08:42:58,535] - [task:2250] INFO -  <<<<< START Executing Query SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2016-04-06 08:42:58,536] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM default WHERE  job_title = "Sales" 
[2016-04-06 08:42:58,539] - [rest_client:2508] INFO - query params : scan_consistency=request_plus&statement=SELECT+%2A+FROM+default+WHERE++job_title+%3D+%22Sales%22+
[2016-04-06 08:42:58,711] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 153.409768ms
[2016-04-06 08:42:58,730] - [tuq_helper:108] INFO -  Analyzing Actual Result
[2016-04-06 08:42:58,731] - [tuq_helper:110] INFO -  Analyzing Expected Result
[2016-04-06 08:42:58,734] - [task:2260] INFO -  <<<<< Done Executing Query SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2016-04-06 08:42:58,734] - [task:2289] INFO -  <<<<< Done VERIFYING Query SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2016-04-06 08:42:59,735] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee5f47e79e4eb942fa9cf6ef6a51526140job_title'
[2016-04-06 08:42:59,739] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee5f47e79e4eb942fa9cf6ef6a51526140job_title%27
[2016-04-06 08:42:59,769] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 29.045033ms
[2016-04-06 08:42:59,770] - [tuq_helper:75] INFO - RUN QUERY DROP INDEX default.employee5f47e79e4eb942fa9cf6ef6a51526140job_title USING GSI
[2016-04-06 08:42:59,772] - [rest_client:2508] INFO - query params : statement=DROP+INDEX+default.employee5f47e79e4eb942fa9cf6ef6a51526140job_title+USING+GSI
[2016-04-06 08:42:59,790] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 14.214498ms
[2016-04-06 08:42:59,790] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee5f47e79e4eb942fa9cf6ef6a51526140job_title'
[2016-04-06 08:42:59,795] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee5f47e79e4eb942fa9cf6ef6a51526140job_title%27
[2016-04-06 08:42:59,824] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 26.819345ms
[2016-04-06 08:42:59,829] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:42:59,829] - [remote_util:168] INFO - connecting to 127.0.0.1 with username : Administrator password : asdasd ssh_key: 
[2016-04-06 08:42:59,829] - [remote_util:201] INFO - Connected to 127.0.0.1
[2016-04-06 08:42:59,836] - [rest_client:1222] INFO - /diag/eval status on 127.0.0.1:9000: True content: "/opt/build/ns_server/logs/n_0" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))).
[2016-04-06 08:42:59,980] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: zgrep "panic" "/opt/build/ns_server/logs/n_0"/indexer.log* | wc -l
[2016-04-06 08:43:00,004] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:43:00,011] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:43:00,012] - [tuq_helper:392] INFO - CHECK FOR PRIMARY INDEXES
[2016-04-06 08:43:00,012] - [tuq_helper:399] INFO - DROP PRIMARY INDEX ON default USING VIEW 
[2016-04-06 08:43:00,012] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2016-04-06 08:43:00,015] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2016-04-06 08:43:00,036] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 20.240352ms
[2016-04-06 08:43:00,037] - [tuq_helper:75] INFO - RUN QUERY DROP PRIMARY INDEX ON default USING VIEW 
[2016-04-06 08:43:00,039] - [rest_client:2508] INFO - query params : statement=DROP+PRIMARY+INDEX+ON+default+USING+VIEW+
[2016-04-06 08:43:00,056] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 14.923223ms
[2016-04-06 08:43:00,207] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: rm -f /opt/couchbase/var/lib/couchbase/meta;rm -f /tmp/log_upr_client.sock
[2016-04-06 08:43:00,228] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:43:00,229] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: killall cbq-engine
[2016-04-06 08:43:00,255] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:43:00,256] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: killall tuqtng
[2016-04-06 08:43:00,291] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:43:00,291] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: killall indexer
[2016-04-06 08:43:00,323] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:43:00,324] - [basetestcase:297] INFO - ==============  basetestcase cleanup was started for test #3 test_multi_create_query_explain_drop_index ==============
[2016-04-06 08:43:00,347] - [bucket_helper:137] INFO - deleting existing buckets [u'default'] on 127.0.0.1
[2016-04-06 08:43:00,347] - [bucket_helper:139] INFO - remove bucket default ...
[2016-04-06 08:43:01,543] - [bucket_helper:153] INFO - deleted bucket : default from 127.0.0.1
[2016-04-06 08:43:01,544] - [bucket_helper:226] INFO - waiting for bucket deletion to complete....
[2016-04-06 08:43:01,552] - [rest_client:127] INFO - node 127.0.0.1 existing buckets : []
[2016-04-06 08:43:01,566] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:43:01,571] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:43:01,576] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:43:01,603] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9000
[2016-04-06 08:43:01,617] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9000 is running
[2016-04-06 08:43:01,620] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9001
[2016-04-06 08:43:01,622] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9001 is running
[2016-04-06 08:43:01,626] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9002
[2016-04-06 08:43:01,629] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9002 is running
[2016-04-06 08:43:01,631] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9003
[2016-04-06 08:43:01,634] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9003 is running
[2016-04-06 08:43:01,634] - [basetestcase:315] INFO - ==============  basetestcase cleanup was finished for test #3 test_multi_create_query_explain_drop_index ==============
ok

----------------------------------------------------------------------
Ran 1 test in 34.523s

OK
test_multi_create_query_explain_drop_index (2i.indexscans_2i.SecondaryIndexingScanTests) ... Cluster instance shutdown with force
summary so far suite 2i.indexscans_2i.SecondaryIndexingScanTests , pass 3 , fail 0
testrunner logs, diags and results are available under /opt/build/testrunner/logs/testrunner-16-Apr-06_08-41-08/test_3
Logs will be stored at /opt/build/testrunner/logs/testrunner-16-Apr-06_08-41-08/test_4

./testrunner -i b/resources/dev-4-nodes-xdcr_n1ql_2i.ini makefile=True -t 2i.indexscans_2i.SecondaryIndexingScanTests.test_multi_create_query_explain_drop_index,groups=simple:equals:no_orderby_groupby:range,dataset=default,doc-per-day=1,use_gsi_for_primary=False,use_gsi_for_secondary=False

Test Input params:
{'doc-per-day': '1', 'use_gsi_for_primary': 'False', 'use_gsi_for_secondary': 'False', 'makefile': 'True', 'conf_file': 'conf/simple_2i_n1ql.conf', 'dataset': 'default', 'cluster_name': 'dev-4-nodes-xdcr_n1ql_2i', 'ini': 'b/resources/dev-4-nodes-xdcr_n1ql_2i.ini', 'groups': 'simple:equals:no_orderby_groupby:range', 'case_number': 4, 'num_nodes': 4, 'logs_folder': '/opt/build/testrunner/logs/testrunner-16-Apr-06_08-41-08/test_4', 'spec': 'simple_2i_n1ql'}
[2016-04-06 08:43:01,661] - [rest_client:1636] INFO - Node versions in cluster [u'4.5.35007-1459908767-rel-enterprise']
[2016-04-06 08:43:01,661] - [basetestcase:146] INFO - ==============  basetestcase setup was started for test #4 test_multi_create_query_explain_drop_index==============
[2016-04-06 08:43:01,674] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:43:01,684] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:43:01,689] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:43:01,694] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:43:01,703] - [basetestcase:365] INFO - sleep for 10 secs.  ...
[2016-04-06 08:43:11,707] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9000
[2016-04-06 08:43:11,709] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9000 is running
[2016-04-06 08:43:11,712] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9001
[2016-04-06 08:43:11,715] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9001 is running
[2016-04-06 08:43:11,717] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9002
[2016-04-06 08:43:11,720] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9002 is running
[2016-04-06 08:43:11,723] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9003
[2016-04-06 08:43:11,726] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9003 is running
[2016-04-06 08:43:11,726] - [basetestcase:167] INFO - initializing cluster
[2016-04-06 08:43:12,673] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:43:12,676] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:43:12,680] - [rest_client:789] INFO - /node/controller/setupServices params on 127.0.0.1: 9000:services=kv%2Cindex%2Cn1ql&password=asdasd&hostname=127.0.0.1%3A9000&user=Administrator
[2016-04-06 08:43:12,697] - [rest_client:757] ERROR - http://127.0.0.1:9000//node/controller/setupServices error 400 reason: unknown ["cannot change node services after cluster is provisioned"]
[2016-04-06 08:43:12,697] - [rest_client:794] INFO - This node is already provisioned with services, we do not consider this as failure for test case
[2016-04-06 08:43:12,697] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9000:username=Administrator&password=asdasd&port=9000
[2016-04-06 08:43:12,716] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:43:12,721] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:43:12,725] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9001:username=Administrator&password=asdasd&port=9001
[2016-04-06 08:43:12,741] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:43:12,744] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:43:12,748] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9002:username=Administrator&password=asdasd&port=9002
[2016-04-06 08:43:12,763] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:43:12,766] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:43:12,772] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9003:username=Administrator&password=asdasd&port=9003
[2016-04-06 08:43:12,788] - [basetestcase:187] INFO - done initializing cluster
[2016-04-06 08:43:13,796] - [rest_client:1772] INFO - http://127.0.0.1:9000/pools/default/buckets with param: bucketType=membase&evictionPolicy=valueOnly&threadsNumber=3&ramQuotaMB=8041&proxyPort=11211&authType=sasl&name=default&flushEnabled=1&replicaNumber=1&replicaIndex=1&saslPassword=None
[2016-04-06 08:43:13,803] - [rest_client:1794] INFO - 0.01 seconds to create bucket default
[2016-04-06 08:43:13,803] - [bucket_helper:305] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2016-04-06 08:43:14,882] - [data_helper:295] INFO - creating direct client 127.0.0.1:12000 default
[2016-04-06 08:43:14,913] - [data_helper:295] INFO - creating direct client 127.0.0.1:12000 default
[2016-04-06 08:43:14,937] - [task:288] INFO - bucket 'default' was created with per node RAM quota: 8041
[2016-04-06 08:43:14,962] - [basetestcase:257] INFO - ==============  basetestcase setup was finished for test #4 test_multi_create_query_explain_drop_index ==============
[2016-04-06 08:43:14,968] - [basetestcase:365] INFO - sleep for 10 secs.  ...
[2016-04-06 08:43:24,980] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:43:24,981] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:43:24,987] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:43:24,987] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:43:24,993] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:43:24,993] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:43:24,999] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:43:24,999] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:43:25,008] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:43:25,008] - [remote_util:168] INFO - connecting to 127.0.0.1 with username : Administrator password : asdasd ssh_key: 
[2016-04-06 08:43:25,009] - [remote_util:201] INFO - Connected to 127.0.0.1
[2016-04-06 08:43:25,948] - [basetestcase:2084] INFO - create 2016 to default documents...
[2016-04-06 08:43:25,965] - [data_helper:295] INFO - creating direct client 127.0.0.1:12000 default
[2016-04-06 08:43:28,137] - [basetestcase:2095] INFO - LOAD IS FINISHED
[2016-04-06 08:43:28,148] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:43:28,148] - [newtuq:92] INFO - ip:127.0.0.1 port:9000 ssh_username:Administrator
number of buckets 1
[2016-04-06 08:43:28,148] - [tuq_helper:419] INFO - CREATE PRIMARY INDEX ON default  USING VIEW 
[2016-04-06 08:43:28,148] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2016-04-06 08:43:28,151] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2016-04-06 08:43:28,179] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 26.098999ms
[2016-04-06 08:43:28,179] - [tuq_helper:75] INFO - RUN QUERY CREATE PRIMARY INDEX ON default  USING VIEW 
[2016-04-06 08:43:28,184] - [rest_client:2508] INFO - query params : statement=CREATE+PRIMARY+INDEX+ON+default++USING+VIEW+
[2016-04-06 08:43:33,212] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 5.026862597s
[2016-04-06 08:43:33,212] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2016-04-06 08:43:33,215] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2016-04-06 08:43:33,236] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 19.193751ms
[2016-04-06 08:43:33,237] - [base_2i:48] INFO - {'in_between': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}, 'initial': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}, 'after': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}, 'before': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}}
[2016-04-06 08:43:33,251] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:43:33,263] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:43:33,961] - [tuq_helper:75] INFO - RUN QUERY CREATE INDEX employee761eb09fdf904a278d0b2e5ba4dcf1f5job_title ON default(job_title) USING VIEW 
[2016-04-06 08:43:33,964] - [rest_client:2508] INFO - query params : statement=CREATE+INDEX+employee761eb09fdf904a278d0b2e5ba4dcf1f5job_title+ON+default%28job_title%29+USING+VIEW+
[2016-04-06 08:43:34,286] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 320.234582ms
[2016-04-06 08:43:34,286] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee761eb09fdf904a278d0b2e5ba4dcf1f5job_title'
[2016-04-06 08:43:34,289] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee761eb09fdf904a278d0b2e5ba4dcf1f5job_title%27
[2016-04-06 08:43:34,311] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 20.42803ms
[2016-04-06 08:43:35,312] - [task:2250] INFO -  <<<<< START Executing Query EXPLAIN SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2016-04-06 08:43:35,312] - [tuq_helper:75] INFO - RUN QUERY EXPLAIN SELECT * FROM default WHERE  job_title = "Sales" 
[2016-04-06 08:43:35,316] - [rest_client:2508] INFO - query params : statement=EXPLAIN+SELECT+%2A+FROM+default+WHERE++job_title+%3D+%22Sales%22+
[2016-04-06 08:43:35,319] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 1.432542ms
[2016-04-06 08:43:35,319] - [task:2259] INFO - {u'status': u'success', u'metrics': {u'elapsedTime': u'1.432542ms', u'executionTime': u'1.361268ms', u'resultSize': 2443, u'resultCount': 1}, u'results': [{u'text': u'SELECT * FROM default WHERE  job_title = "Sales"', u'plan': {u'#operator': u'Sequence', u'~children': [{u'index': u'employee761eb09fdf904a278d0b2e5ba4dcf1f5job_title', u'namespace': u'default', u'keyspace': u'default', u'#operator': u'IndexScan', u'spans': [{u'Range': {u'High': [u'"Sales"'], u'Low': [u'"Sales"'], u'Inclusion': 3}}], u'using': u'view', u'index_id': u'employee761eb09fdf904a278d0b2e5ba4dcf1f5job_title'}, {u'#operator': u'Parallel', u'~child': {u'#operator': u'Sequence', u'~children': [{u'keyspace': u'default', u'#operator': u'Fetch', u'namespace': u'default'}, {u'#operator': u'Filter', u'condition': u'((`default`.`job_title`) = "Sales")'}, {u'#operator': u'InitialProject', u'result_terms': [{u'expr': u'self', u'star': True}]}, {u'#operator': u'FinalProject'}]}}]}}], u'requestID': u'6fdb023e-4d99-48a4-83bc-db95a2201f25', u'signature': u'json'}
[2016-04-06 08:43:35,319] - [task:2260] INFO -  <<<<< Done Executing Query EXPLAIN SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2016-04-06 08:43:35,319] - [task:2289] INFO -  <<<<< Done VERIFYING Query EXPLAIN SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2016-04-06 08:43:35,319] - [base_2i:294] INFO - Query : SELECT * FROM default WHERE  job_title = "Sales" 
[2016-04-06 08:43:35,320] - [tuq_generators:66] INFO - FROM clause ===== is default
[2016-04-06 08:43:35,320] - [tuq_generators:68] INFO - WHERE clause ===== is   doc["job_title"] == "Sales" 
[2016-04-06 08:43:35,320] - [tuq_generators:70] INFO - UNNEST clause ===== is None
[2016-04-06 08:43:35,320] - [tuq_generators:72] INFO - SELECT clause ===== is {"*" : doc,}
[2016-04-06 08:43:36,320] - [task:2250] INFO -  <<<<< START Executing Query SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2016-04-06 08:43:36,321] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM default WHERE  job_title = "Sales" 
[2016-04-06 08:43:36,324] - [rest_client:2508] INFO - query params : scan_consistency=request_plus&statement=SELECT+%2A+FROM+default+WHERE++job_title+%3D+%22Sales%22+
[2016-04-06 08:43:36,512] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 174.226847ms
[2016-04-06 08:43:36,530] - [tuq_helper:108] INFO -  Analyzing Actual Result
[2016-04-06 08:43:36,531] - [tuq_helper:110] INFO -  Analyzing Expected Result
[2016-04-06 08:43:36,533] - [task:2260] INFO -  <<<<< Done Executing Query SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2016-04-06 08:43:36,533] - [task:2289] INFO -  <<<<< Done VERIFYING Query SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2016-04-06 08:43:37,534] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee761eb09fdf904a278d0b2e5ba4dcf1f5job_title'
[2016-04-06 08:43:37,537] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee761eb09fdf904a278d0b2e5ba4dcf1f5job_title%27
[2016-04-06 08:43:37,568] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 28.783695ms
[2016-04-06 08:43:37,568] - [tuq_helper:75] INFO - RUN QUERY DROP INDEX default.employee761eb09fdf904a278d0b2e5ba4dcf1f5job_title USING VIEW 
[2016-04-06 08:43:37,571] - [rest_client:2508] INFO - query params : statement=DROP+INDEX+default.employee761eb09fdf904a278d0b2e5ba4dcf1f5job_title+USING+VIEW+
[2016-04-06 08:43:37,579] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 6.072864ms
[2016-04-06 08:43:37,579] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee761eb09fdf904a278d0b2e5ba4dcf1f5job_title'
[2016-04-06 08:43:37,582] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee761eb09fdf904a278d0b2e5ba4dcf1f5job_title%27
[2016-04-06 08:43:37,643] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 60.01424ms
[2016-04-06 08:43:37,649] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:43:37,649] - [remote_util:168] INFO - connecting to 127.0.0.1 with username : Administrator password : asdasd ssh_key: 
[2016-04-06 08:43:37,649] - [remote_util:201] INFO - Connected to 127.0.0.1
[2016-04-06 08:43:37,659] - [rest_client:1222] INFO - /diag/eval status on 127.0.0.1:9000: True content: "/opt/build/ns_server/logs/n_0" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))).
[2016-04-06 08:43:37,807] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: zgrep "panic" "/opt/build/ns_server/logs/n_0"/indexer.log* | wc -l
[2016-04-06 08:43:37,832] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:43:37,838] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:43:37,839] - [tuq_helper:392] INFO - CHECK FOR PRIMARY INDEXES
[2016-04-06 08:43:37,839] - [tuq_helper:399] INFO - DROP PRIMARY INDEX ON default USING VIEW 
[2016-04-06 08:43:37,839] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2016-04-06 08:43:37,842] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2016-04-06 08:43:37,896] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 52.171761ms
[2016-04-06 08:43:37,896] - [tuq_helper:75] INFO - RUN QUERY DROP PRIMARY INDEX ON default USING VIEW 
[2016-04-06 08:43:37,899] - [rest_client:2508] INFO - query params : statement=DROP+PRIMARY+INDEX+ON+default+USING+VIEW+
[2016-04-06 08:43:37,947] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 45.128643ms
[2016-04-06 08:43:38,087] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: rm -f /opt/couchbase/var/lib/couchbase/meta;rm -f /tmp/log_upr_client.sock
[2016-04-06 08:43:38,109] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:43:38,109] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: killall cbq-engine
[2016-04-06 08:43:38,134] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:43:38,134] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: killall tuqtng
[2016-04-06 08:43:38,168] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:43:38,169] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: killall indexer
[2016-04-06 08:43:38,205] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:43:38,206] - [basetestcase:297] INFO - ==============  basetestcase cleanup was started for test #4 test_multi_create_query_explain_drop_index ==============
[2016-04-06 08:43:38,226] - [bucket_helper:137] INFO - deleting existing buckets [u'default'] on 127.0.0.1
[2016-04-06 08:43:38,226] - [bucket_helper:139] INFO - remove bucket default ...
[2016-04-06 08:43:40,068] - [bucket_helper:153] INFO - deleted bucket : default from 127.0.0.1
[2016-04-06 08:43:40,068] - [bucket_helper:226] INFO - waiting for bucket deletion to complete....
[2016-04-06 08:43:40,070] - [rest_client:127] INFO - node 127.0.0.1 existing buckets : []
[2016-04-06 08:43:40,077] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:43:40,087] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:43:40,096] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:43:40,109] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9000
[2016-04-06 08:43:40,112] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9000 is running
[2016-04-06 08:43:40,114] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9001
[2016-04-06 08:43:40,117] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9001 is running
[2016-04-06 08:43:40,120] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9002
[2016-04-06 08:43:40,122] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9002 is running
[2016-04-06 08:43:40,125] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9003
[2016-04-06 08:43:40,128] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9003 is running
[2016-04-06 08:43:40,128] - [basetestcase:315] INFO - ==============  basetestcase cleanup was finished for test #4 test_multi_create_query_explain_drop_index ==============
ok

----------------------------------------------------------------------
Ran 1 test in 38.478s

OK
test_multi_create_query_explain_drop_index (2i.indexscans_2i.SecondaryIndexingScanTests) ... Cluster instance shutdown with force
summary so far suite 2i.indexscans_2i.SecondaryIndexingScanTests , pass 4 , fail 0
testrunner logs, diags and results are available under /opt/build/testrunner/logs/testrunner-16-Apr-06_08-41-08/test_4
Logs will be stored at /opt/build/testrunner/logs/testrunner-16-Apr-06_08-41-08/test_5

./testrunner -i b/resources/dev-4-nodes-xdcr_n1ql_2i.ini makefile=True -t 2i.indexscans_2i.SecondaryIndexingScanTests.test_multi_create_query_explain_drop_index,groups=simple:equals:no_orderby_groupby:range,dataset=default,doc-per-day=1,use_gsi_for_primary=True,use_gsi_for_secondary=True,doc_ops=True,delete_ops_per=.5,run_async=True,scan_consistency=request_plus

Test Input params:
{'doc-per-day': '1', 'use_gsi_for_primary': 'True', 'use_gsi_for_secondary': 'True', 'doc_ops': 'True', 'makefile': 'True', 'delete_ops_per': '.5', 'conf_file': 'conf/simple_2i_n1ql.conf', 'dataset': 'default', 'cluster_name': 'dev-4-nodes-xdcr_n1ql_2i', 'num_nodes': 4, 'ini': 'b/resources/dev-4-nodes-xdcr_n1ql_2i.ini', 'groups': 'simple:equals:no_orderby_groupby:range', 'case_number': 5, 'scan_consistency': 'request_plus', 'run_async': 'True', 'logs_folder': '/opt/build/testrunner/logs/testrunner-16-Apr-06_08-41-08/test_5', 'spec': 'simple_2i_n1ql'}
[2016-04-06 08:43:40,154] - [rest_client:1636] INFO - Node versions in cluster [u'4.5.35007-1459908767-rel-enterprise']
[2016-04-06 08:43:40,154] - [basetestcase:146] INFO - ==============  basetestcase setup was started for test #5 test_multi_create_query_explain_drop_index==============
[2016-04-06 08:43:40,171] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:43:40,175] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:43:40,180] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:43:40,185] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:43:40,194] - [basetestcase:365] INFO - sleep for 10 secs.  ...
[2016-04-06 08:43:50,208] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9000
[2016-04-06 08:43:50,210] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9000 is running
[2016-04-06 08:43:50,213] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9001
[2016-04-06 08:43:50,216] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9001 is running
[2016-04-06 08:43:50,219] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9002
[2016-04-06 08:43:50,222] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9002 is running
[2016-04-06 08:43:50,225] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9003
[2016-04-06 08:43:50,228] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9003 is running
[2016-04-06 08:43:50,229] - [basetestcase:167] INFO - initializing cluster
[2016-04-06 08:43:51,165] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:43:51,168] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:43:51,174] - [rest_client:789] INFO - /node/controller/setupServices params on 127.0.0.1: 9000:services=kv%2Cindex%2Cn1ql&password=asdasd&hostname=127.0.0.1%3A9000&user=Administrator
[2016-04-06 08:43:51,184] - [rest_client:757] ERROR - http://127.0.0.1:9000//node/controller/setupServices error 400 reason: unknown ["cannot change node services after cluster is provisioned"]
[2016-04-06 08:43:51,184] - [rest_client:794] INFO - This node is already provisioned with services, we do not consider this as failure for test case
[2016-04-06 08:43:51,185] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9000:username=Administrator&password=asdasd&port=9000
[2016-04-06 08:43:51,205] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:43:51,208] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:43:51,214] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9001:username=Administrator&password=asdasd&port=9001
[2016-04-06 08:43:51,233] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:43:51,237] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:43:51,241] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9002:username=Administrator&password=asdasd&port=9002
[2016-04-06 08:43:51,259] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:43:51,265] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:43:51,270] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9003:username=Administrator&password=asdasd&port=9003
[2016-04-06 08:43:51,281] - [basetestcase:187] INFO - done initializing cluster
[2016-04-06 08:43:52,290] - [rest_client:1772] INFO - http://127.0.0.1:9000/pools/default/buckets with param: bucketType=membase&evictionPolicy=valueOnly&threadsNumber=3&ramQuotaMB=8041&proxyPort=11211&authType=sasl&name=default&flushEnabled=1&replicaNumber=1&replicaIndex=1&saslPassword=None
[2016-04-06 08:43:52,298] - [rest_client:1794] INFO - 0.01 seconds to create bucket default
[2016-04-06 08:43:52,298] - [bucket_helper:305] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2016-04-06 08:43:53,366] - [data_helper:295] INFO - creating direct client 127.0.0.1:12000 default
[2016-04-06 08:43:53,395] - [data_helper:295] INFO - creating direct client 127.0.0.1:12000 default
[2016-04-06 08:43:53,425] - [task:288] INFO - bucket 'default' was created with per node RAM quota: 8041
[2016-04-06 08:43:53,455] - [basetestcase:257] INFO - ==============  basetestcase setup was finished for test #5 test_multi_create_query_explain_drop_index ==============
[2016-04-06 08:43:53,460] - [basetestcase:365] INFO - sleep for 10 secs.  ...
[2016-04-06 08:44:03,468] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:44:03,468] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:44:03,474] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:44:03,474] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:44:03,480] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:44:03,481] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:44:03,487] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:44:03,487] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:44:03,492] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:44:03,492] - [remote_util:168] INFO - connecting to 127.0.0.1 with username : Administrator password : asdasd ssh_key: 
[2016-04-06 08:44:03,492] - [remote_util:201] INFO - Connected to 127.0.0.1
[2016-04-06 08:44:04,423] - [basetestcase:2084] INFO - create 2016 to default documents...
[2016-04-06 08:44:04,441] - [data_helper:295] INFO - creating direct client 127.0.0.1:12000 default
[2016-04-06 08:44:06,566] - [basetestcase:2095] INFO - LOAD IS FINISHED
[2016-04-06 08:44:06,571] - [newtuq:82] INFO - {'remaining': {'start': 0, 'end': 1}, 'delete': {'start': 0, 'end': 0}}
[2016-04-06 08:44:07,150] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:44:07,150] - [newtuq:92] INFO - ip:127.0.0.1 port:9000 ssh_username:Administrator
number of buckets 1
[2016-04-06 08:44:07,150] - [tuq_helper:419] INFO - CREATE PRIMARY INDEX ON default  USING GSI
[2016-04-06 08:44:07,150] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2016-04-06 08:44:07,153] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2016-04-06 08:44:07,183] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 28.339506ms
[2016-04-06 08:44:07,183] - [tuq_helper:75] INFO - RUN QUERY CREATE PRIMARY INDEX ON default  USING GSI
[2016-04-06 08:44:07,187] - [rest_client:2508] INFO - query params : statement=CREATE+PRIMARY+INDEX+ON+default++USING+GSI
[2016-04-06 08:44:07,731] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 541.748542ms
[2016-04-06 08:44:07,732] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2016-04-06 08:44:07,735] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2016-04-06 08:44:07,751] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 13.943883ms
[2016-04-06 08:44:07,752] - [base_2i:48] INFO - {'in_between': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}, 'initial': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}, 'after': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}, 'before': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}}
[2016-04-06 08:44:07,768] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:44:07,781] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:44:08,445] - [tuq_helper:75] INFO - RUN QUERY CREATE INDEX employee24f211c53b024826bb2a19256a8ae525job_title ON default(job_title) USING GSI  WITH {'nodes': ['127.0.0.1:9000'], 'defer_build': True}
[2016-04-06 08:44:08,449] - [rest_client:2508] INFO - query params : statement=CREATE+INDEX+employee24f211c53b024826bb2a19256a8ae525job_title+ON+default%28job_title%29+USING+GSI++WITH+%7B%27nodes%27%3A+%5B%27127.0.0.1%3A9000%27%5D%2C+%27defer_build%27%3A+True%7D
[2016-04-06 08:44:08,480] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 30.056043ms
[2016-04-06 08:44:08,481] - [base_2i:102] INFO - BUILD INDEX on default(employee24f211c53b024826bb2a19256a8ae525job_title) USING GSI
[2016-04-06 08:44:09,481] - [tuq_helper:75] INFO - RUN QUERY BUILD INDEX on default(employee24f211c53b024826bb2a19256a8ae525job_title) USING GSI
[2016-04-06 08:44:09,484] - [rest_client:2508] INFO - query params : statement=BUILD+INDEX+on+default%28employee24f211c53b024826bb2a19256a8ae525job_title%29+USING+GSI
[2016-04-06 08:44:09,498] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 12.378985ms
[2016-04-06 08:44:09,499] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee24f211c53b024826bb2a19256a8ae525job_title'
[2016-04-06 08:44:09,503] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee24f211c53b024826bb2a19256a8ae525job_title%27
[2016-04-06 08:44:09,525] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 20.968481ms
[2016-04-06 08:44:10,526] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee24f211c53b024826bb2a19256a8ae525job_title'
[2016-04-06 08:44:10,530] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee24f211c53b024826bb2a19256a8ae525job_title%27
[2016-04-06 08:44:10,556] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 24.973934ms
[2016-04-06 08:44:11,558] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee24f211c53b024826bb2a19256a8ae525job_title'
[2016-04-06 08:44:11,561] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee24f211c53b024826bb2a19256a8ae525job_title%27
[2016-04-06 08:44:11,579] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 16.829377ms
[2016-04-06 08:44:12,047] - [basetestcase:2084] INFO - delete 0 to default documents...
[2016-04-06 08:44:12,065] - [data_helper:295] INFO - creating direct client 127.0.0.1:12000 default
[2016-04-06 08:44:12,582] - [basetestcase:2095] INFO - LOAD IS FINISHED
[2016-04-06 08:44:13,586] - [task:2250] INFO -  <<<<< START Executing Query EXPLAIN SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2016-04-06 08:44:13,586] - [tuq_helper:75] INFO - RUN QUERY EXPLAIN SELECT * FROM default WHERE  job_title = "Sales" 
[2016-04-06 08:44:13,590] - [rest_client:2508] INFO - query params : statement=EXPLAIN+SELECT+%2A+FROM+default+WHERE++job_title+%3D+%22Sales%22+
[2016-04-06 08:44:13,596] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 4.635647ms
[2016-04-06 08:44:13,596] - [task:2259] INFO - {u'status': u'success', u'metrics': {u'elapsedTime': u'4.635647ms', u'executionTime': u'4.570372ms', u'resultSize': 2409, u'resultCount': 1}, u'results': [{u'text': u'SELECT * FROM default WHERE  job_title = "Sales"', u'plan': {u'#operator': u'Sequence', u'~children': [{u'index': u'employee24f211c53b024826bb2a19256a8ae525job_title', u'namespace': u'default', u'keyspace': u'default', u'#operator': u'IndexScan', u'spans': [{u'Range': {u'High': [u'"Sales"'], u'Low': [u'"Sales"'], u'Inclusion': 3}}], u'using': u'gsi', u'index_id': u'c75370d96619748c'}, {u'#operator': u'Parallel', u'~child': {u'#operator': u'Sequence', u'~children': [{u'keyspace': u'default', u'#operator': u'Fetch', u'namespace': u'default'}, {u'#operator': u'Filter', u'condition': u'((`default`.`job_title`) = "Sales")'}, {u'#operator': u'InitialProject', u'result_terms': [{u'expr': u'self', u'star': True}]}, {u'#operator': u'FinalProject'}]}}]}}], u'requestID': u'b5df3c2c-8ed8-4464-a687-f0cbf5a90bd9', u'signature': u'json'}
[2016-04-06 08:44:13,596] - [task:2260] INFO -  <<<<< Done Executing Query EXPLAIN SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2016-04-06 08:44:13,596] - [task:2289] INFO -  <<<<< Done VERIFYING Query EXPLAIN SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2016-04-06 08:44:13,597] - [base_2i:294] INFO - Query : SELECT * FROM default WHERE  job_title = "Sales" 
[2016-04-06 08:44:13,597] - [tuq_generators:66] INFO - FROM clause ===== is default
[2016-04-06 08:44:13,597] - [tuq_generators:68] INFO - WHERE clause ===== is   doc["job_title"] == "Sales" 
[2016-04-06 08:44:13,597] - [tuq_generators:70] INFO - UNNEST clause ===== is None
[2016-04-06 08:44:13,598] - [tuq_generators:72] INFO - SELECT clause ===== is {"*" : doc,}
[2016-04-06 08:44:14,597] - [task:2250] INFO -  <<<<< START Executing Query SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2016-04-06 08:44:14,597] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM default WHERE  job_title = "Sales" 
[2016-04-06 08:44:14,600] - [rest_client:2508] INFO - query params : scan_consistency=request_plus&statement=SELECT+%2A+FROM+default+WHERE++job_title+%3D+%22Sales%22+
[2016-04-06 08:44:14,759] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 140.363744ms
[2016-04-06 08:44:14,778] - [tuq_helper:108] INFO -  Analyzing Actual Result
[2016-04-06 08:44:14,778] - [tuq_helper:110] INFO -  Analyzing Expected Result
[2016-04-06 08:44:14,781] - [task:2260] INFO -  <<<<< Done Executing Query SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2016-04-06 08:44:14,781] - [task:2289] INFO -  <<<<< Done VERIFYING Query SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2016-04-06 08:44:15,782] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee24f211c53b024826bb2a19256a8ae525job_title'
[2016-04-06 08:44:15,786] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee24f211c53b024826bb2a19256a8ae525job_title%27
[2016-04-06 08:44:15,802] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 14.639339ms
[2016-04-06 08:44:15,802] - [tuq_helper:75] INFO - RUN QUERY DROP INDEX default.employee24f211c53b024826bb2a19256a8ae525job_title USING GSI
[2016-04-06 08:44:15,805] - [rest_client:2508] INFO - query params : statement=DROP+INDEX+default.employee24f211c53b024826bb2a19256a8ae525job_title+USING+GSI
[2016-04-06 08:44:15,820] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 13.536622ms
[2016-04-06 08:44:15,820] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee24f211c53b024826bb2a19256a8ae525job_title'
[2016-04-06 08:44:15,825] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee24f211c53b024826bb2a19256a8ae525job_title%27
[2016-04-06 08:44:15,842] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 14.792653ms
[2016-04-06 08:44:15,847] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:44:15,847] - [remote_util:168] INFO - connecting to 127.0.0.1 with username : Administrator password : asdasd ssh_key: 
[2016-04-06 08:44:15,847] - [remote_util:201] INFO - Connected to 127.0.0.1
[2016-04-06 08:44:15,858] - [rest_client:1222] INFO - /diag/eval status on 127.0.0.1:9000: True content: "/opt/build/ns_server/logs/n_0" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))).
[2016-04-06 08:44:16,059] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: zgrep "panic" "/opt/build/ns_server/logs/n_0"/indexer.log* | wc -l
[2016-04-06 08:44:16,090] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:44:16,097] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:44:16,097] - [tuq_helper:392] INFO - CHECK FOR PRIMARY INDEXES
[2016-04-06 08:44:16,097] - [tuq_helper:399] INFO - DROP PRIMARY INDEX ON default USING GSI
[2016-04-06 08:44:16,097] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2016-04-06 08:44:16,100] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2016-04-06 08:44:16,114] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 12.24871ms
[2016-04-06 08:44:16,114] - [tuq_helper:75] INFO - RUN QUERY DROP PRIMARY INDEX ON default USING GSI
[2016-04-06 08:44:16,117] - [rest_client:2508] INFO - query params : statement=DROP+PRIMARY+INDEX+ON+default+USING+GSI
[2016-04-06 08:44:16,134] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 15.059191ms
[2016-04-06 08:44:16,331] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: rm -f /opt/couchbase/var/lib/couchbase/meta;rm -f /tmp/log_upr_client.sock
[2016-04-06 08:44:16,359] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:44:16,360] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: killall cbq-engine
[2016-04-06 08:44:16,390] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:44:16,391] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: killall tuqtng
[2016-04-06 08:44:16,434] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:44:16,435] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: killall indexer
[2016-04-06 08:44:16,471] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:44:16,472] - [basetestcase:297] INFO - ==============  basetestcase cleanup was started for test #5 test_multi_create_query_explain_drop_index ==============
[2016-04-06 08:44:16,492] - [bucket_helper:137] INFO - deleting existing buckets [u'default'] on 127.0.0.1
[2016-04-06 08:44:16,492] - [bucket_helper:139] INFO - remove bucket default ...
[2016-04-06 08:44:18,567] - [bucket_helper:153] INFO - deleted bucket : default from 127.0.0.1
[2016-04-06 08:44:18,567] - [bucket_helper:226] INFO - waiting for bucket deletion to complete....
[2016-04-06 08:44:18,569] - [rest_client:127] INFO - node 127.0.0.1 existing buckets : []
[2016-04-06 08:44:18,575] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:44:18,580] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:44:18,585] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:44:18,599] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9000
[2016-04-06 08:44:18,602] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9000 is running
[2016-04-06 08:44:18,604] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9001
[2016-04-06 08:44:18,607] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9001 is running
[2016-04-06 08:44:18,610] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9002
[2016-04-06 08:44:18,613] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9002 is running
[2016-04-06 08:44:18,615] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9003
[2016-04-06 08:44:18,618] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9003 is running
[2016-04-06 08:44:18,618] - [basetestcase:315] INFO - ==============  basetestcase cleanup was finished for test #5 test_multi_create_query_explain_drop_index ==============
ok

----------------------------------------------------------------------
Ran 1 test in 38.480s

OK
test_multi_create_query_explain_drop_index (2i.indexscans_2i.SecondaryIndexingScanTests) ... Cluster instance shutdown with force
summary so far suite 2i.indexscans_2i.SecondaryIndexingScanTests , pass 5 , fail 0
testrunner logs, diags and results are available under /opt/build/testrunner/logs/testrunner-16-Apr-06_08-41-08/test_5
Logs will be stored at /opt/build/testrunner/logs/testrunner-16-Apr-06_08-41-08/test_6

./testrunner -i b/resources/dev-4-nodes-xdcr_n1ql_2i.ini makefile=True -t 2i.indexscans_2i.SecondaryIndexingScanTests.test_multi_create_query_explain_drop_index,groups=simple:equals:no_orderby_groupby:range,dataset=default,doc-per-day=1,use_gsi_for_primary=True,use_gsi_for_secondary=True,doc_ops=True,update_ops_per=.5,run_async=True,scan_consistency=request_plus

Test Input params:
{'doc-per-day': '1', 'use_gsi_for_primary': 'True', 'use_gsi_for_secondary': 'True', 'doc_ops': 'True', 'makefile': 'True', 'conf_file': 'conf/simple_2i_n1ql.conf', 'dataset': 'default', 'cluster_name': 'dev-4-nodes-xdcr_n1ql_2i', 'num_nodes': 4, 'update_ops_per': '.5', 'ini': 'b/resources/dev-4-nodes-xdcr_n1ql_2i.ini', 'groups': 'simple:equals:no_orderby_groupby:range', 'case_number': 6, 'scan_consistency': 'request_plus', 'run_async': 'True', 'logs_folder': '/opt/build/testrunner/logs/testrunner-16-Apr-06_08-41-08/test_6', 'spec': 'simple_2i_n1ql'}
[2016-04-06 08:44:18,672] - [rest_client:1636] INFO - Node versions in cluster [u'4.5.35007-1459908767-rel-enterprise']
[2016-04-06 08:44:18,672] - [basetestcase:146] INFO - ==============  basetestcase setup was started for test #6 test_multi_create_query_explain_drop_index==============
[2016-04-06 08:44:18,683] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:44:18,688] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:44:18,693] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:44:18,697] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:44:18,711] - [basetestcase:365] INFO - sleep for 10 secs.  ...
[2016-04-06 08:44:28,717] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9000
[2016-04-06 08:44:28,720] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9000 is running
[2016-04-06 08:44:28,723] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9001
[2016-04-06 08:44:28,726] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9001 is running
[2016-04-06 08:44:28,728] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9002
[2016-04-06 08:44:28,731] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9002 is running
[2016-04-06 08:44:28,734] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9003
[2016-04-06 08:44:28,736] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9003 is running
[2016-04-06 08:44:28,736] - [basetestcase:167] INFO - initializing cluster
[2016-04-06 08:44:29,680] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:44:29,683] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:44:29,687] - [rest_client:789] INFO - /node/controller/setupServices params on 127.0.0.1: 9000:services=kv%2Cindex%2Cn1ql&password=asdasd&hostname=127.0.0.1%3A9000&user=Administrator
[2016-04-06 08:44:29,700] - [rest_client:757] ERROR - http://127.0.0.1:9000//node/controller/setupServices error 400 reason: unknown ["cannot change node services after cluster is provisioned"]
[2016-04-06 08:44:29,700] - [rest_client:794] INFO - This node is already provisioned with services, we do not consider this as failure for test case
[2016-04-06 08:44:29,700] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9000:username=Administrator&password=asdasd&port=9000
[2016-04-06 08:44:29,728] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:44:29,731] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:44:29,735] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9001:username=Administrator&password=asdasd&port=9001
[2016-04-06 08:44:29,752] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:44:29,756] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:44:29,760] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9002:username=Administrator&password=asdasd&port=9002
[2016-04-06 08:44:29,782] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:44:29,785] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:44:29,789] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9003:username=Administrator&password=asdasd&port=9003
[2016-04-06 08:44:29,799] - [basetestcase:187] INFO - done initializing cluster
[2016-04-06 08:44:30,808] - [rest_client:1772] INFO - http://127.0.0.1:9000/pools/default/buckets with param: bucketType=membase&evictionPolicy=valueOnly&threadsNumber=3&ramQuotaMB=8041&proxyPort=11211&authType=sasl&name=default&flushEnabled=1&replicaNumber=1&replicaIndex=1&saslPassword=None
[2016-04-06 08:44:30,813] - [rest_client:1794] INFO - 0.00 seconds to create bucket default
[2016-04-06 08:44:30,813] - [bucket_helper:305] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2016-04-06 08:44:31,891] - [data_helper:295] INFO - creating direct client 127.0.0.1:12000 default
[2016-04-06 08:44:31,927] - [data_helper:295] INFO - creating direct client 127.0.0.1:12000 default
[2016-04-06 08:44:31,954] - [task:288] INFO - bucket 'default' was created with per node RAM quota: 8041
[2016-04-06 08:44:31,973] - [basetestcase:257] INFO - ==============  basetestcase setup was finished for test #6 test_multi_create_query_explain_drop_index ==============
[2016-04-06 08:44:31,980] - [basetestcase:365] INFO - sleep for 10 secs.  ...
[2016-04-06 08:44:41,993] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:44:41,993] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:44:41,999] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:44:41,999] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:44:42,005] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:44:42,005] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:44:42,011] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:44:42,011] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:44:42,017] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:44:42,017] - [remote_util:168] INFO - connecting to 127.0.0.1 with username : Administrator password : asdasd ssh_key: 
[2016-04-06 08:44:42,017] - [remote_util:201] INFO - Connected to 127.0.0.1
[2016-04-06 08:44:42,953] - [basetestcase:2084] INFO - create 2016 to default documents...
[2016-04-06 08:44:42,973] - [data_helper:295] INFO - creating direct client 127.0.0.1:12000 default
[2016-04-06 08:44:45,146] - [basetestcase:2095] INFO - LOAD IS FINISHED
[2016-04-06 08:44:45,151] - [newtuq:82] INFO - {'update': {'start': 0, 'end': 0}, 'remaining': {'start': 0, 'end': 1}}
[2016-04-06 08:44:45,937] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:44:45,937] - [newtuq:92] INFO - ip:127.0.0.1 port:9000 ssh_username:Administrator
number of buckets 1
[2016-04-06 08:44:45,937] - [tuq_helper:419] INFO - CREATE PRIMARY INDEX ON default  USING GSI
[2016-04-06 08:44:45,937] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2016-04-06 08:44:45,940] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2016-04-06 08:44:45,975] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 33.252453ms
[2016-04-06 08:44:45,975] - [tuq_helper:75] INFO - RUN QUERY CREATE PRIMARY INDEX ON default  USING GSI
[2016-04-06 08:44:45,978] - [rest_client:2508] INFO - query params : statement=CREATE+PRIMARY+INDEX+ON+default++USING+GSI
[2016-04-06 08:44:46,495] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 514.789226ms
[2016-04-06 08:44:46,495] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2016-04-06 08:44:46,498] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2016-04-06 08:44:46,513] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 13.342894ms
[2016-04-06 08:44:46,514] - [base_2i:48] INFO - {'in_between': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}, 'initial': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}, 'after': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}, 'before': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}}
[2016-04-06 08:44:46,528] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:44:46,532] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:44:46,975] - [tuq_helper:75] INFO - RUN QUERY CREATE INDEX employeedda8933edfdd4619ab641c8b80bc116ejob_title ON default(job_title) USING GSI  WITH {'nodes': ['127.0.0.1:9000'], 'defer_build': True}
[2016-04-06 08:44:46,978] - [rest_client:2508] INFO - query params : statement=CREATE+INDEX+employeedda8933edfdd4619ab641c8b80bc116ejob_title+ON+default%28job_title%29+USING+GSI++WITH+%7B%27nodes%27%3A+%5B%27127.0.0.1%3A9000%27%5D%2C+%27defer_build%27%3A+True%7D
[2016-04-06 08:44:47,008] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 28.113975ms
[2016-04-06 08:44:47,008] - [base_2i:102] INFO - BUILD INDEX on default(employeedda8933edfdd4619ab641c8b80bc116ejob_title) USING GSI
[2016-04-06 08:44:48,010] - [tuq_helper:75] INFO - RUN QUERY BUILD INDEX on default(employeedda8933edfdd4619ab641c8b80bc116ejob_title) USING GSI
[2016-04-06 08:44:48,013] - [rest_client:2508] INFO - query params : statement=BUILD+INDEX+on+default%28employeedda8933edfdd4619ab641c8b80bc116ejob_title%29+USING+GSI
[2016-04-06 08:44:48,028] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 12.991685ms
[2016-04-06 08:44:49,029] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employeedda8933edfdd4619ab641c8b80bc116ejob_title'
[2016-04-06 08:44:49,032] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employeedda8933edfdd4619ab641c8b80bc116ejob_title%27
[2016-04-06 08:44:49,048] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 14.137319ms
[2016-04-06 08:44:50,049] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employeedda8933edfdd4619ab641c8b80bc116ejob_title'
[2016-04-06 08:44:50,052] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employeedda8933edfdd4619ab641c8b80bc116ejob_title%27
[2016-04-06 08:44:50,074] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 20.143888ms
[2016-04-06 08:44:50,527] - [basetestcase:2084] INFO - update 0 to default documents...
[2016-04-06 08:44:50,551] - [data_helper:295] INFO - creating direct client 127.0.0.1:12000 default
[2016-04-06 08:44:51,077] - [basetestcase:2095] INFO - LOAD IS FINISHED
[2016-04-06 08:44:52,082] - [task:2250] INFO -  <<<<< START Executing Query EXPLAIN SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2016-04-06 08:44:52,083] - [tuq_helper:75] INFO - RUN QUERY EXPLAIN SELECT * FROM default WHERE  job_title = "Sales" 
[2016-04-06 08:44:52,086] - [rest_client:2508] INFO - query params : statement=EXPLAIN+SELECT+%2A+FROM+default+WHERE++job_title+%3D+%22Sales%22+
[2016-04-06 08:44:52,091] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 3.342891ms
[2016-04-06 08:44:52,091] - [task:2259] INFO - {u'status': u'success', u'metrics': {u'elapsedTime': u'3.342891ms', u'executionTime': u'3.285135ms', u'resultSize': 2409, u'resultCount': 1}, u'results': [{u'text': u'SELECT * FROM default WHERE  job_title = "Sales"', u'plan': {u'#operator': u'Sequence', u'~children': [{u'index': u'employeedda8933edfdd4619ab641c8b80bc116ejob_title', u'namespace': u'default', u'keyspace': u'default', u'#operator': u'IndexScan', u'spans': [{u'Range': {u'High': [u'"Sales"'], u'Low': [u'"Sales"'], u'Inclusion': 3}}], u'using': u'gsi', u'index_id': u'bb8d31b657f0c994'}, {u'#operator': u'Parallel', u'~child': {u'#operator': u'Sequence', u'~children': [{u'keyspace': u'default', u'#operator': u'Fetch', u'namespace': u'default'}, {u'#operator': u'Filter', u'condition': u'((`default`.`job_title`) = "Sales")'}, {u'#operator': u'InitialProject', u'result_terms': [{u'expr': u'self', u'star': True}]}, {u'#operator': u'FinalProject'}]}}]}}], u'requestID': u'd33009bf-089a-4d89-932a-ed303dd35e05', u'signature': u'json'}
[2016-04-06 08:44:52,091] - [task:2260] INFO -  <<<<< Done Executing Query EXPLAIN SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2016-04-06 08:44:52,091] - [task:2289] INFO -  <<<<< Done VERIFYING Query EXPLAIN SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2016-04-06 08:44:52,092] - [base_2i:294] INFO - Query : SELECT * FROM default WHERE  job_title = "Sales" 
[2016-04-06 08:44:52,092] - [tuq_generators:66] INFO - FROM clause ===== is default
[2016-04-06 08:44:52,092] - [tuq_generators:68] INFO - WHERE clause ===== is   doc["job_title"] == "Sales" 
[2016-04-06 08:44:52,092] - [tuq_generators:70] INFO - UNNEST clause ===== is None
[2016-04-06 08:44:52,092] - [tuq_generators:72] INFO - SELECT clause ===== is {"*" : doc,}
[2016-04-06 08:44:53,093] - [task:2250] INFO -  <<<<< START Executing Query SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2016-04-06 08:44:53,093] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM default WHERE  job_title = "Sales" 
[2016-04-06 08:44:53,096] - [rest_client:2508] INFO - query params : scan_consistency=request_plus&statement=SELECT+%2A+FROM+default+WHERE++job_title+%3D+%22Sales%22+
[2016-04-06 08:44:53,271] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 160.537793ms
[2016-04-06 08:44:53,291] - [tuq_helper:108] INFO -  Analyzing Actual Result
[2016-04-06 08:44:53,291] - [tuq_helper:110] INFO -  Analyzing Expected Result
[2016-04-06 08:44:53,294] - [task:2260] INFO -  <<<<< Done Executing Query SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2016-04-06 08:44:53,294] - [task:2289] INFO -  <<<<< Done VERIFYING Query SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2016-04-06 08:44:54,296] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employeedda8933edfdd4619ab641c8b80bc116ejob_title'
[2016-04-06 08:44:54,299] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employeedda8933edfdd4619ab641c8b80bc116ejob_title%27
[2016-04-06 08:44:54,319] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 17.807988ms
[2016-04-06 08:44:54,319] - [tuq_helper:75] INFO - RUN QUERY DROP INDEX default.employeedda8933edfdd4619ab641c8b80bc116ejob_title USING GSI
[2016-04-06 08:44:54,321] - [rest_client:2508] INFO - query params : statement=DROP+INDEX+default.employeedda8933edfdd4619ab641c8b80bc116ejob_title+USING+GSI
[2016-04-06 08:44:54,338] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 14.825837ms
[2016-04-06 08:44:54,338] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employeedda8933edfdd4619ab641c8b80bc116ejob_title'
[2016-04-06 08:44:54,341] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employeedda8933edfdd4619ab641c8b80bc116ejob_title%27
[2016-04-06 08:44:54,356] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 13.384212ms
[2016-04-06 08:44:54,361] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:44:54,361] - [remote_util:168] INFO - connecting to 127.0.0.1 with username : Administrator password : asdasd ssh_key: 
[2016-04-06 08:44:54,361] - [remote_util:201] INFO - Connected to 127.0.0.1
[2016-04-06 08:44:54,366] - [rest_client:1222] INFO - /diag/eval status on 127.0.0.1:9000: True content: "/opt/build/ns_server/logs/n_0" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))).
[2016-04-06 08:44:54,564] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: zgrep "panic" "/opt/build/ns_server/logs/n_0"/indexer.log* | wc -l
[2016-04-06 08:44:54,594] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:44:54,601] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:44:54,601] - [tuq_helper:392] INFO - CHECK FOR PRIMARY INDEXES
[2016-04-06 08:44:54,601] - [tuq_helper:399] INFO - DROP PRIMARY INDEX ON default USING GSI
[2016-04-06 08:44:54,601] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2016-04-06 08:44:54,604] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2016-04-06 08:44:54,625] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 18.570112ms
[2016-04-06 08:44:54,625] - [tuq_helper:75] INFO - RUN QUERY DROP PRIMARY INDEX ON default USING GSI
[2016-04-06 08:44:54,628] - [rest_client:2508] INFO - query params : statement=DROP+PRIMARY+INDEX+ON+default+USING+GSI
[2016-04-06 08:44:54,646] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 16.427962ms
[2016-04-06 08:44:54,840] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: rm -f /opt/couchbase/var/lib/couchbase/meta;rm -f /tmp/log_upr_client.sock
[2016-04-06 08:44:54,869] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:44:54,870] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: killall cbq-engine
[2016-04-06 08:44:54,901] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:44:54,902] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: killall tuqtng
[2016-04-06 08:44:54,950] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:44:54,951] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: killall indexer
[2016-04-06 08:44:55,008] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:44:55,009] - [basetestcase:297] INFO - ==============  basetestcase cleanup was started for test #6 test_multi_create_query_explain_drop_index ==============
[2016-04-06 08:44:55,040] - [bucket_helper:137] INFO - deleting existing buckets [u'default'] on 127.0.0.1
[2016-04-06 08:44:55,040] - [bucket_helper:139] INFO - remove bucket default ...
[2016-04-06 08:44:57,040] - [bucket_helper:153] INFO - deleted bucket : default from 127.0.0.1
[2016-04-06 08:44:57,041] - [bucket_helper:226] INFO - waiting for bucket deletion to complete....
[2016-04-06 08:44:57,043] - [rest_client:127] INFO - node 127.0.0.1 existing buckets : []
[2016-04-06 08:44:57,049] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:44:57,055] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:44:57,060] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:44:57,073] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9000
[2016-04-06 08:44:57,080] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9000 is running
[2016-04-06 08:44:57,083] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9001
[2016-04-06 08:44:57,086] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9001 is running
[2016-04-06 08:44:57,089] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9002
[2016-04-06 08:44:57,091] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9002 is running
[2016-04-06 08:44:57,094] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9003
[2016-04-06 08:44:57,096] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9003 is running
[2016-04-06 08:44:57,096] - [basetestcase:315] INFO - ==============  basetestcase cleanup was finished for test #6 test_multi_create_query_explain_drop_index ==============
ok

----------------------------------------------------------------------
Ran 1 test in 38.441s

OK
test_multi_create_query_explain_drop_index (2i.indexscans_2i.SecondaryIndexingScanTests) ... Cluster instance shutdown with force
summary so far suite 2i.indexscans_2i.SecondaryIndexingScanTests , pass 6 , fail 0
testrunner logs, diags and results are available under /opt/build/testrunner/logs/testrunner-16-Apr-06_08-41-08/test_6
Logs will be stored at /opt/build/testrunner/logs/testrunner-16-Apr-06_08-41-08/test_7

./testrunner -i b/resources/dev-4-nodes-xdcr_n1ql_2i.ini makefile=True -t 2i.indexscans_2i.SecondaryIndexingScanTests.test_multi_create_query_explain_drop_index,groups=simple:equals:no_orderby_groupby:range,dataset=default,doc-per-day=1,use_gsi_for_primary=True,use_gsi_for_secondary=True,doc_ops=True,expiry_ops_per=.5,run_async=True,scan_consistency=request_plus

Test Input params:
{'doc-per-day': '1', 'use_gsi_for_primary': 'True', 'use_gsi_for_secondary': 'True', 'doc_ops': 'True', 'case_number': 7, 'makefile': 'True', 'conf_file': 'conf/simple_2i_n1ql.conf', 'dataset': 'default', 'cluster_name': 'dev-4-nodes-xdcr_n1ql_2i', 'expiry_ops_per': '.5', 'ini': 'b/resources/dev-4-nodes-xdcr_n1ql_2i.ini', 'groups': 'simple:equals:no_orderby_groupby:range', 'num_nodes': 4, 'scan_consistency': 'request_plus', 'run_async': 'True', 'logs_folder': '/opt/build/testrunner/logs/testrunner-16-Apr-06_08-41-08/test_7', 'spec': 'simple_2i_n1ql'}
[2016-04-06 08:44:57,148] - [rest_client:1636] INFO - Node versions in cluster [u'4.5.35007-1459908767-rel-enterprise']
[2016-04-06 08:44:57,148] - [basetestcase:146] INFO - ==============  basetestcase setup was started for test #7 test_multi_create_query_explain_drop_index==============
[2016-04-06 08:44:57,161] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:44:57,166] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:44:57,170] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:44:57,175] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:44:57,188] - [basetestcase:365] INFO - sleep for 10 secs.  ...
[2016-04-06 08:45:07,201] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9000
[2016-04-06 08:45:07,203] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9000 is running
[2016-04-06 08:45:07,206] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9001
[2016-04-06 08:45:07,208] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9001 is running
[2016-04-06 08:45:07,211] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9002
[2016-04-06 08:45:07,214] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9002 is running
[2016-04-06 08:45:07,217] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9003
[2016-04-06 08:45:07,220] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9003 is running
[2016-04-06 08:45:07,221] - [basetestcase:167] INFO - initializing cluster
[2016-04-06 08:45:08,160] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:45:08,163] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:45:08,179] - [rest_client:789] INFO - /node/controller/setupServices params on 127.0.0.1: 9000:services=kv%2Cindex%2Cn1ql&password=asdasd&hostname=127.0.0.1%3A9000&user=Administrator
[2016-04-06 08:45:08,183] - [rest_client:757] ERROR - http://127.0.0.1:9000//node/controller/setupServices error 400 reason: unknown ["cannot change node services after cluster is provisioned"]
[2016-04-06 08:45:08,183] - [rest_client:794] INFO - This node is already provisioned with services, we do not consider this as failure for test case
[2016-04-06 08:45:08,184] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9000:username=Administrator&password=asdasd&port=9000
[2016-04-06 08:45:08,199] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:45:08,203] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:45:08,206] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9001:username=Administrator&password=asdasd&port=9001
[2016-04-06 08:45:08,229] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:45:08,233] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:45:08,237] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9002:username=Administrator&password=asdasd&port=9002
[2016-04-06 08:45:08,258] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:45:08,262] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:45:08,266] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9003:username=Administrator&password=asdasd&port=9003
[2016-04-06 08:45:08,279] - [basetestcase:187] INFO - done initializing cluster
[2016-04-06 08:45:09,289] - [rest_client:1772] INFO - http://127.0.0.1:9000/pools/default/buckets with param: bucketType=membase&evictionPolicy=valueOnly&threadsNumber=3&ramQuotaMB=8041&proxyPort=11211&authType=sasl&name=default&flushEnabled=1&replicaNumber=1&replicaIndex=1&saslPassword=None
[2016-04-06 08:45:09,301] - [rest_client:1794] INFO - 0.01 seconds to create bucket default
[2016-04-06 08:45:09,302] - [bucket_helper:305] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2016-04-06 08:45:10,366] - [data_helper:295] INFO - creating direct client 127.0.0.1:12000 default
[2016-04-06 08:45:10,399] - [data_helper:295] INFO - creating direct client 127.0.0.1:12000 default
[2016-04-06 08:45:10,431] - [task:288] INFO - bucket 'default' was created with per node RAM quota: 8041
[2016-04-06 08:45:10,455] - [basetestcase:257] INFO - ==============  basetestcase setup was finished for test #7 test_multi_create_query_explain_drop_index ==============
[2016-04-06 08:45:10,459] - [basetestcase:365] INFO - sleep for 10 secs.  ...
[2016-04-06 08:45:20,475] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:45:20,476] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:45:20,481] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:45:20,481] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:45:20,487] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:45:20,487] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:45:20,492] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:45:20,493] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:45:20,498] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:45:20,498] - [remote_util:168] INFO - connecting to 127.0.0.1 with username : Administrator password : asdasd ssh_key: 
[2016-04-06 08:45:20,498] - [remote_util:201] INFO - Connected to 127.0.0.1
[2016-04-06 08:45:21,423] - [basetestcase:2084] INFO - create 2016 to default documents...
[2016-04-06 08:45:21,441] - [data_helper:295] INFO - creating direct client 127.0.0.1:12000 default
[2016-04-06 08:45:23,624] - [basetestcase:2095] INFO - LOAD IS FINISHED
[2016-04-06 08:45:23,629] - [newtuq:82] INFO - {'remaining': {'start': 0, 'end': 1}, 'expiry': {'start': 0, 'end': 0}}
[2016-04-06 08:45:24,198] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:45:24,198] - [newtuq:92] INFO - ip:127.0.0.1 port:9000 ssh_username:Administrator
number of buckets 1
[2016-04-06 08:45:24,198] - [tuq_helper:419] INFO - CREATE PRIMARY INDEX ON default  USING GSI
[2016-04-06 08:45:24,198] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2016-04-06 08:45:24,201] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2016-04-06 08:45:24,230] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 27.519964ms
[2016-04-06 08:45:24,231] - [tuq_helper:75] INFO - RUN QUERY CREATE PRIMARY INDEX ON default  USING GSI
[2016-04-06 08:45:24,237] - [rest_client:2508] INFO - query params : statement=CREATE+PRIMARY+INDEX+ON+default++USING+GSI
[2016-04-06 08:45:24,745] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 505.926714ms
[2016-04-06 08:45:24,745] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2016-04-06 08:45:24,748] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2016-04-06 08:45:24,764] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 13.85802ms
[2016-04-06 08:45:24,765] - [base_2i:48] INFO - {'in_between': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}, 'initial': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}, 'after': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}, 'before': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}}
[2016-04-06 08:45:24,782] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:45:24,787] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:45:25,450] - [tuq_helper:75] INFO - RUN QUERY CREATE INDEX employeeb0fab38a9127497a8e37e0991f2655a3job_title ON default(job_title) USING GSI  WITH {'nodes': ['127.0.0.1:9000'], 'defer_build': True}
[2016-04-06 08:45:25,453] - [rest_client:2508] INFO - query params : statement=CREATE+INDEX+employeeb0fab38a9127497a8e37e0991f2655a3job_title+ON+default%28job_title%29+USING+GSI++WITH+%7B%27nodes%27%3A+%5B%27127.0.0.1%3A9000%27%5D%2C+%27defer_build%27%3A+True%7D
[2016-04-06 08:45:25,492] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 37.557528ms
[2016-04-06 08:45:25,492] - [base_2i:102] INFO - BUILD INDEX on default(employeeb0fab38a9127497a8e37e0991f2655a3job_title) USING GSI
[2016-04-06 08:45:26,494] - [tuq_helper:75] INFO - RUN QUERY BUILD INDEX on default(employeeb0fab38a9127497a8e37e0991f2655a3job_title) USING GSI
[2016-04-06 08:45:26,497] - [rest_client:2508] INFO - query params : statement=BUILD+INDEX+on+default%28employeeb0fab38a9127497a8e37e0991f2655a3job_title%29+USING+GSI
[2016-04-06 08:45:26,509] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 10.762744ms
[2016-04-06 08:45:27,510] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employeeb0fab38a9127497a8e37e0991f2655a3job_title'
[2016-04-06 08:45:27,513] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employeeb0fab38a9127497a8e37e0991f2655a3job_title%27
[2016-04-06 08:45:27,535] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 20.157668ms
[2016-04-06 08:45:28,536] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employeeb0fab38a9127497a8e37e0991f2655a3job_title'
[2016-04-06 08:45:28,539] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employeeb0fab38a9127497a8e37e0991f2655a3job_title%27
[2016-04-06 08:45:28,561] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 20.273762ms
[2016-04-06 08:45:29,015] - [basetestcase:2084] INFO - update 0 to default documents...
[2016-04-06 08:45:29,033] - [data_helper:295] INFO - creating direct client 127.0.0.1:12000 default
[2016-04-06 08:45:29,564] - [basetestcase:2095] INFO - LOAD IS FINISHED
[2016-04-06 08:45:29,585] - [data_helper:292] INFO - dict:{'username': 'Administrator', 'ip': '127.0.0.1', 'password': 'asdasd', 'port': u'9000'}
[2016-04-06 08:45:29,586] - [data_helper:293] INFO - creating direct client 127.0.0.1:12000 default
[2016-04-06 08:45:29,603] - [cluster_helper:341] INFO - Setting flush param on server {'username': 'Administrator', 'ip': '127.0.0.1', 'password': 'asdasd', 'port': u'9000'}, exp_pager_stime to 10 on default
setting param: exp_pager_stime 10
[2016-04-06 08:45:29,604] - [cluster_helper:355] INFO - Setting flush param on server {'username': 'Administrator', 'ip': '127.0.0.1', 'password': 'asdasd', 'port': u'9000'}, exp_pager_stime to 10, result: (843512511, 0, '')
[2016-04-06 08:45:30,568] - [task:2250] INFO -  <<<<< START Executing Query EXPLAIN SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2016-04-06 08:45:30,568] - [tuq_helper:75] INFO - RUN QUERY EXPLAIN SELECT * FROM default WHERE  job_title = "Sales" 
[2016-04-06 08:45:30,572] - [rest_client:2508] INFO - query params : statement=EXPLAIN+SELECT+%2A+FROM+default+WHERE++job_title+%3D+%22Sales%22+
[2016-04-06 08:45:30,577] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 3.415951ms
[2016-04-06 08:45:30,578] - [task:2259] INFO - {u'status': u'success', u'metrics': {u'elapsedTime': u'3.415951ms', u'executionTime': u'3.358839ms', u'resultSize': 2409, u'resultCount': 1}, u'results': [{u'text': u'SELECT * FROM default WHERE  job_title = "Sales"', u'plan': {u'#operator': u'Sequence', u'~children': [{u'index': u'employeeb0fab38a9127497a8e37e0991f2655a3job_title', u'namespace': u'default', u'keyspace': u'default', u'#operator': u'IndexScan', u'spans': [{u'Range': {u'High': [u'"Sales"'], u'Low': [u'"Sales"'], u'Inclusion': 3}}], u'using': u'gsi', u'index_id': u'5a730e68871bf717'}, {u'#operator': u'Parallel', u'~child': {u'#operator': u'Sequence', u'~children': [{u'keyspace': u'default', u'#operator': u'Fetch', u'namespace': u'default'}, {u'#operator': u'Filter', u'condition': u'((`default`.`job_title`) = "Sales")'}, {u'#operator': u'InitialProject', u'result_terms': [{u'expr': u'self', u'star': True}]}, {u'#operator': u'FinalProject'}]}}]}}], u'requestID': u'643b014d-2737-48fc-bf9a-c666b19a220f', u'signature': u'json'}
[2016-04-06 08:45:30,578] - [task:2260] INFO -  <<<<< Done Executing Query EXPLAIN SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2016-04-06 08:45:30,578] - [task:2289] INFO -  <<<<< Done VERIFYING Query EXPLAIN SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2016-04-06 08:45:30,578] - [base_2i:294] INFO - Query : SELECT * FROM default WHERE  job_title = "Sales" 
[2016-04-06 08:45:30,579] - [tuq_generators:66] INFO - FROM clause ===== is default
[2016-04-06 08:45:30,579] - [tuq_generators:68] INFO - WHERE clause ===== is   doc["job_title"] == "Sales" 
[2016-04-06 08:45:30,579] - [tuq_generators:70] INFO - UNNEST clause ===== is None
[2016-04-06 08:45:30,579] - [tuq_generators:72] INFO - SELECT clause ===== is {"*" : doc,}
[2016-04-06 08:45:31,579] - [task:2250] INFO -  <<<<< START Executing Query SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2016-04-06 08:45:31,580] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM default WHERE  job_title = "Sales" 
[2016-04-06 08:45:31,583] - [rest_client:2508] INFO - query params : scan_consistency=request_plus&statement=SELECT+%2A+FROM+default+WHERE++job_title+%3D+%22Sales%22+
[2016-04-06 08:45:31,748] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 149.301802ms
[2016-04-06 08:45:31,766] - [tuq_helper:108] INFO -  Analyzing Actual Result
[2016-04-06 08:45:31,766] - [tuq_helper:110] INFO -  Analyzing Expected Result
[2016-04-06 08:45:31,769] - [task:2260] INFO -  <<<<< Done Executing Query SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2016-04-06 08:45:31,769] - [task:2289] INFO -  <<<<< Done VERIFYING Query SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2016-04-06 08:45:32,770] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employeeb0fab38a9127497a8e37e0991f2655a3job_title'
[2016-04-06 08:45:32,774] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employeeb0fab38a9127497a8e37e0991f2655a3job_title%27
[2016-04-06 08:45:32,798] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 21.55816ms
[2016-04-06 08:45:32,798] - [tuq_helper:75] INFO - RUN QUERY DROP INDEX default.employeeb0fab38a9127497a8e37e0991f2655a3job_title USING GSI
[2016-04-06 08:45:32,801] - [rest_client:2508] INFO - query params : statement=DROP+INDEX+default.employeeb0fab38a9127497a8e37e0991f2655a3job_title+USING+GSI
[2016-04-06 08:45:32,819] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 16.400524ms
[2016-04-06 08:45:32,819] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employeeb0fab38a9127497a8e37e0991f2655a3job_title'
[2016-04-06 08:45:32,823] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employeeb0fab38a9127497a8e37e0991f2655a3job_title%27
[2016-04-06 08:45:32,839] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 15.008192ms
[2016-04-06 08:45:32,845] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:45:32,845] - [remote_util:168] INFO - connecting to 127.0.0.1 with username : Administrator password : asdasd ssh_key: 
[2016-04-06 08:45:32,845] - [remote_util:201] INFO - Connected to 127.0.0.1
[2016-04-06 08:45:32,850] - [rest_client:1222] INFO - /diag/eval status on 127.0.0.1:9000: True content: "/opt/build/ns_server/logs/n_0" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))).
[2016-04-06 08:45:33,055] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: zgrep "panic" "/opt/build/ns_server/logs/n_0"/indexer.log* | wc -l
[2016-04-06 08:45:33,088] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:45:33,095] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:45:33,095] - [tuq_helper:392] INFO - CHECK FOR PRIMARY INDEXES
[2016-04-06 08:45:33,096] - [tuq_helper:399] INFO - DROP PRIMARY INDEX ON default USING GSI
[2016-04-06 08:45:33,096] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2016-04-06 08:45:33,099] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2016-04-06 08:45:33,114] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 14.08498ms
[2016-04-06 08:45:33,114] - [tuq_helper:75] INFO - RUN QUERY DROP PRIMARY INDEX ON default USING GSI
[2016-04-06 08:45:33,117] - [rest_client:2508] INFO - query params : statement=DROP+PRIMARY+INDEX+ON+default+USING+GSI
[2016-04-06 08:45:33,137] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 18.160949ms
[2016-04-06 08:45:33,333] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: rm -f /opt/couchbase/var/lib/couchbase/meta;rm -f /tmp/log_upr_client.sock
[2016-04-06 08:45:33,362] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:45:33,363] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: killall cbq-engine
[2016-04-06 08:45:33,394] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:45:33,395] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: killall tuqtng
[2016-04-06 08:45:33,439] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:45:33,440] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: killall indexer
[2016-04-06 08:45:33,478] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:45:33,479] - [basetestcase:297] INFO - ==============  basetestcase cleanup was started for test #7 test_multi_create_query_explain_drop_index ==============
[2016-04-06 08:45:33,502] - [bucket_helper:137] INFO - deleting existing buckets [u'default'] on 127.0.0.1
[2016-04-06 08:45:33,502] - [bucket_helper:139] INFO - remove bucket default ...
[2016-04-06 08:45:35,563] - [bucket_helper:153] INFO - deleted bucket : default from 127.0.0.1
[2016-04-06 08:45:35,564] - [bucket_helper:226] INFO - waiting for bucket deletion to complete....
[2016-04-06 08:45:35,566] - [rest_client:127] INFO - node 127.0.0.1 existing buckets : []
[2016-04-06 08:45:35,573] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:45:35,578] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:45:35,584] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:45:35,596] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9000
[2016-04-06 08:45:35,598] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9000 is running
[2016-04-06 08:45:35,601] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9001
[2016-04-06 08:45:35,604] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9001 is running
[2016-04-06 08:45:35,607] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9002
[2016-04-06 08:45:35,610] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9002 is running
[2016-04-06 08:45:35,612] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9003
[2016-04-06 08:45:35,615] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9003 is running
[2016-04-06 08:45:35,615] - [basetestcase:315] INFO - ==============  basetestcase cleanup was finished for test #7 test_multi_create_query_explain_drop_index ==============
ok

----------------------------------------------------------------------
Ran 1 test in 38.481s

OK
test_multi_create_query_explain_drop_index (2i.indexscans_2i.SecondaryIndexingScanTests) ... Cluster instance shutdown with force
summary so far suite 2i.indexscans_2i.SecondaryIndexingScanTests , pass 7 , fail 0
testrunner logs, diags and results are available under /opt/build/testrunner/logs/testrunner-16-Apr-06_08-41-08/test_7
Logs will be stored at /opt/build/testrunner/logs/testrunner-16-Apr-06_08-41-08/test_8

./testrunner -i b/resources/dev-4-nodes-xdcr_n1ql_2i.ini makefile=True -t 2i.indexscans_2i.SecondaryIndexingScanTests.test_multi_create_query_explain_drop_index,groups=simple:equals:no_orderby_groupby:range,dataset=default,doc-per-day=1,use_gsi_for_primary=True,use_gsi_for_secondary=True,doc_ops=True,create_ops_per=.5,run_async=True,scan_consistency=request_plus

Test Input params:
{'doc-per-day': '1', 'use_gsi_for_primary': 'True', 'use_gsi_for_secondary': 'True', 'doc_ops': 'True', 'makefile': 'True', 'conf_file': 'conf/simple_2i_n1ql.conf', 'dataset': 'default', 'cluster_name': 'dev-4-nodes-xdcr_n1ql_2i', 'num_nodes': 4, 'run_async': 'True', 'groups': 'simple:equals:no_orderby_groupby:range', 'case_number': 8, 'scan_consistency': 'request_plus', 'create_ops_per': '.5', 'logs_folder': '/opt/build/testrunner/logs/testrunner-16-Apr-06_08-41-08/test_8', 'spec': 'simple_2i_n1ql', 'ini': 'b/resources/dev-4-nodes-xdcr_n1ql_2i.ini'}
[2016-04-06 08:45:35,663] - [rest_client:1636] INFO - Node versions in cluster [u'4.5.35007-1459908767-rel-enterprise']
[2016-04-06 08:45:35,663] - [basetestcase:146] INFO - ==============  basetestcase setup was started for test #8 test_multi_create_query_explain_drop_index==============
[2016-04-06 08:45:35,675] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:45:35,679] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:45:35,684] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:45:35,689] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:45:35,699] - [basetestcase:365] INFO - sleep for 10 secs.  ...
[2016-04-06 08:45:45,712] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9000
[2016-04-06 08:45:45,715] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9000 is running
[2016-04-06 08:45:45,717] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9001
[2016-04-06 08:45:45,720] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9001 is running
[2016-04-06 08:45:45,723] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9002
[2016-04-06 08:45:45,728] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9002 is running
[2016-04-06 08:45:45,731] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9003
[2016-04-06 08:45:45,734] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9003 is running
[2016-04-06 08:45:45,734] - [basetestcase:167] INFO - initializing cluster
[2016-04-06 08:45:46,675] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:45:46,681] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:45:46,692] - [rest_client:789] INFO - /node/controller/setupServices params on 127.0.0.1: 9000:services=kv%2Cindex%2Cn1ql&password=asdasd&hostname=127.0.0.1%3A9000&user=Administrator
[2016-04-06 08:45:46,700] - [rest_client:757] ERROR - http://127.0.0.1:9000//node/controller/setupServices error 400 reason: unknown ["cannot change node services after cluster is provisioned"]
[2016-04-06 08:45:46,701] - [rest_client:794] INFO - This node is already provisioned with services, we do not consider this as failure for test case
[2016-04-06 08:45:46,701] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9000:username=Administrator&password=asdasd&port=9000
[2016-04-06 08:45:46,722] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:45:46,727] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:45:46,731] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9001:username=Administrator&password=asdasd&port=9001
[2016-04-06 08:45:46,748] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:45:46,751] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:45:46,755] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9002:username=Administrator&password=asdasd&port=9002
[2016-04-06 08:45:46,779] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:45:46,782] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:45:46,786] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9003:username=Administrator&password=asdasd&port=9003
[2016-04-06 08:45:46,808] - [basetestcase:187] INFO - done initializing cluster
[2016-04-06 08:45:47,815] - [rest_client:1772] INFO - http://127.0.0.1:9000/pools/default/buckets with param: bucketType=membase&evictionPolicy=valueOnly&threadsNumber=3&ramQuotaMB=8041&proxyPort=11211&authType=sasl&name=default&flushEnabled=1&replicaNumber=1&replicaIndex=1&saslPassword=None
[2016-04-06 08:45:47,820] - [rest_client:1794] INFO - 0.00 seconds to create bucket default
[2016-04-06 08:45:47,820] - [bucket_helper:305] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2016-04-06 08:45:48,891] - [data_helper:295] INFO - creating direct client 127.0.0.1:12000 default
[2016-04-06 08:45:48,927] - [data_helper:295] INFO - creating direct client 127.0.0.1:12000 default
[2016-04-06 08:45:48,953] - [task:288] INFO - bucket 'default' was created with per node RAM quota: 8041
[2016-04-06 08:45:48,983] - [basetestcase:257] INFO - ==============  basetestcase setup was finished for test #8 test_multi_create_query_explain_drop_index ==============
[2016-04-06 08:45:48,988] - [basetestcase:365] INFO - sleep for 10 secs.  ...
[2016-04-06 08:45:58,996] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:45:58,996] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:45:59,003] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:45:59,003] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:45:59,012] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:45:59,012] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:45:59,019] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:45:59,019] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:45:59,028] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:45:59,028] - [remote_util:168] INFO - connecting to 127.0.0.1 with username : Administrator password : asdasd ssh_key: 
[2016-04-06 08:45:59,028] - [remote_util:201] INFO - Connected to 127.0.0.1
[2016-04-06 08:45:59,950] - [basetestcase:2084] INFO - create 2016 to default documents...
[2016-04-06 08:45:59,967] - [data_helper:295] INFO - creating direct client 127.0.0.1:12000 default
[2016-04-06 08:46:02,148] - [basetestcase:2095] INFO - LOAD IS FINISHED
[2016-04-06 08:46:02,153] - [newtuq:82] INFO - {'create': {'start': 1, 'end': 2.5}, 'remaining': {'start': 0, 'end': 1}}
[2016-04-06 08:46:03,378] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:46:03,378] - [newtuq:92] INFO - ip:127.0.0.1 port:9000 ssh_username:Administrator
number of buckets 1
[2016-04-06 08:46:03,378] - [tuq_helper:419] INFO - CREATE PRIMARY INDEX ON default  USING GSI
[2016-04-06 08:46:03,379] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2016-04-06 08:46:03,381] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2016-04-06 08:46:03,411] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 28.449571ms
[2016-04-06 08:46:03,412] - [tuq_helper:75] INFO - RUN QUERY CREATE PRIMARY INDEX ON default  USING GSI
[2016-04-06 08:46:03,416] - [rest_client:2508] INFO - query params : statement=CREATE+PRIMARY+INDEX+ON+default++USING+GSI
[2016-04-06 08:46:03,926] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 508.709752ms
[2016-04-06 08:46:03,927] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2016-04-06 08:46:03,930] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2016-04-06 08:46:03,948] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 16.548817ms
[2016-04-06 08:46:03,948] - [base_2i:48] INFO - {'in_between': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}, 'initial': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}, 'after': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}, 'before': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}}
[2016-04-06 08:46:03,960] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:46:03,965] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:46:03,973] - [tuq_helper:75] INFO - RUN QUERY CREATE INDEX employee89c1e7daac3a4e0492bbf745c7fce4fcjob_title ON default(job_title) USING GSI  WITH {'nodes': ['127.0.0.1:9000'], 'defer_build': True}
[2016-04-06 08:46:03,976] - [rest_client:2508] INFO - query params : statement=CREATE+INDEX+employee89c1e7daac3a4e0492bbf745c7fce4fcjob_title+ON+default%28job_title%29+USING+GSI++WITH+%7B%27nodes%27%3A+%5B%27127.0.0.1%3A9000%27%5D%2C+%27defer_build%27%3A+True%7D
[2016-04-06 08:46:04,007] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 29.578029ms
[2016-04-06 08:46:04,007] - [base_2i:102] INFO - BUILD INDEX on default(employee89c1e7daac3a4e0492bbf745c7fce4fcjob_title) USING GSI
[2016-04-06 08:46:05,009] - [tuq_helper:75] INFO - RUN QUERY BUILD INDEX on default(employee89c1e7daac3a4e0492bbf745c7fce4fcjob_title) USING GSI
[2016-04-06 08:46:05,012] - [rest_client:2508] INFO - query params : statement=BUILD+INDEX+on+default%28employee89c1e7daac3a4e0492bbf745c7fce4fcjob_title%29+USING+GSI
[2016-04-06 08:46:05,027] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 12.697294ms
[2016-04-06 08:46:06,028] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee89c1e7daac3a4e0492bbf745c7fce4fcjob_title'
[2016-04-06 08:46:06,031] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee89c1e7daac3a4e0492bbf745c7fce4fcjob_title%27
[2016-04-06 08:46:06,051] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 18.705005ms
[2016-04-06 08:46:07,053] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee89c1e7daac3a4e0492bbf745c7fce4fcjob_title'
[2016-04-06 08:46:07,056] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee89c1e7daac3a4e0492bbf745c7fce4fcjob_title%27
[2016-04-06 08:46:07,072] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 14.850839ms
[2016-04-06 08:46:07,527] - [basetestcase:2084] INFO - create 3024.0 to default documents...
[2016-04-06 08:46:07,549] - [data_helper:295] INFO - creating direct client 127.0.0.1:12000 default
[2016-04-06 08:46:11,581] - [basetestcase:2095] INFO - LOAD IS FINISHED
[2016-04-06 08:46:12,082] - [task:2250] INFO -  <<<<< START Executing Query EXPLAIN SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2016-04-06 08:46:12,083] - [tuq_helper:75] INFO - RUN QUERY EXPLAIN SELECT * FROM default WHERE  job_title = "Sales" 
[2016-04-06 08:46:12,085] - [rest_client:2508] INFO - query params : statement=EXPLAIN+SELECT+%2A+FROM+default+WHERE++job_title+%3D+%22Sales%22+
[2016-04-06 08:46:12,090] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 3.163646ms
[2016-04-06 08:46:12,090] - [task:2259] INFO - {u'status': u'success', u'metrics': {u'elapsedTime': u'3.163646ms', u'executionTime': u'3.093314ms', u'resultSize': 2409, u'resultCount': 1}, u'results': [{u'text': u'SELECT * FROM default WHERE  job_title = "Sales"', u'plan': {u'#operator': u'Sequence', u'~children': [{u'index': u'employee89c1e7daac3a4e0492bbf745c7fce4fcjob_title', u'namespace': u'default', u'keyspace': u'default', u'#operator': u'IndexScan', u'spans': [{u'Range': {u'High': [u'"Sales"'], u'Low': [u'"Sales"'], u'Inclusion': 3}}], u'using': u'gsi', u'index_id': u'b582b3702d934de9'}, {u'#operator': u'Parallel', u'~child': {u'#operator': u'Sequence', u'~children': [{u'keyspace': u'default', u'#operator': u'Fetch', u'namespace': u'default'}, {u'#operator': u'Filter', u'condition': u'((`default`.`job_title`) = "Sales")'}, {u'#operator': u'InitialProject', u'result_terms': [{u'expr': u'self', u'star': True}]}, {u'#operator': u'FinalProject'}]}}]}}], u'requestID': u'4a7c05e9-5ff8-4500-8b6d-cb8d6bf99f22', u'signature': u'json'}
[2016-04-06 08:46:12,091] - [task:2260] INFO -  <<<<< Done Executing Query EXPLAIN SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2016-04-06 08:46:12,091] - [task:2289] INFO -  <<<<< Done VERIFYING Query EXPLAIN SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2016-04-06 08:46:12,091] - [base_2i:294] INFO - Query : SELECT * FROM default WHERE  job_title = "Sales" 
[2016-04-06 08:46:12,091] - [tuq_generators:66] INFO - FROM clause ===== is default
[2016-04-06 08:46:12,091] - [tuq_generators:68] INFO - WHERE clause ===== is   doc["job_title"] == "Sales" 
[2016-04-06 08:46:12,092] - [tuq_generators:70] INFO - UNNEST clause ===== is None
[2016-04-06 08:46:12,092] - [tuq_generators:72] INFO - SELECT clause ===== is {"*" : doc,}
[2016-04-06 08:46:13,092] - [task:2250] INFO -  <<<<< START Executing Query SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2016-04-06 08:46:13,093] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM default WHERE  job_title = "Sales" 
[2016-04-06 08:46:13,095] - [rest_client:2508] INFO - query params : scan_consistency=request_plus&statement=SELECT+%2A+FROM+default+WHERE++job_title+%3D+%22Sales%22+
[2016-04-06 08:46:13,623] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 483.360734ms
[2016-04-06 08:46:13,676] - [tuq_helper:108] INFO -  Analyzing Actual Result
[2016-04-06 08:46:13,677] - [tuq_helper:110] INFO -  Analyzing Expected Result
[2016-04-06 08:46:13,689] - [task:2260] INFO -  <<<<< Done Executing Query SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2016-04-06 08:46:13,689] - [task:2289] INFO -  <<<<< Done VERIFYING Query SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2016-04-06 08:46:14,691] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee89c1e7daac3a4e0492bbf745c7fce4fcjob_title'
[2016-04-06 08:46:14,695] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee89c1e7daac3a4e0492bbf745c7fce4fcjob_title%27
[2016-04-06 08:46:14,718] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 21.770264ms
[2016-04-06 08:46:14,718] - [tuq_helper:75] INFO - RUN QUERY DROP INDEX default.employee89c1e7daac3a4e0492bbf745c7fce4fcjob_title USING GSI
[2016-04-06 08:46:14,721] - [rest_client:2508] INFO - query params : statement=DROP+INDEX+default.employee89c1e7daac3a4e0492bbf745c7fce4fcjob_title+USING+GSI
[2016-04-06 08:46:14,748] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 25.081954ms
[2016-04-06 08:46:14,748] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee89c1e7daac3a4e0492bbf745c7fce4fcjob_title'
[2016-04-06 08:46:14,751] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee89c1e7daac3a4e0492bbf745c7fce4fcjob_title%27
[2016-04-06 08:46:14,768] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 15.005654ms
[2016-04-06 08:46:14,780] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:46:14,780] - [remote_util:168] INFO - connecting to 127.0.0.1 with username : Administrator password : asdasd ssh_key: 
[2016-04-06 08:46:14,781] - [remote_util:201] INFO - Connected to 127.0.0.1
[2016-04-06 08:46:14,786] - [rest_client:1222] INFO - /diag/eval status on 127.0.0.1:9000: True content: "/opt/build/ns_server/logs/n_0" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))).
[2016-04-06 08:46:15,035] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: zgrep "panic" "/opt/build/ns_server/logs/n_0"/indexer.log* | wc -l
[2016-04-06 08:46:15,072] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:46:15,079] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:46:15,080] - [tuq_helper:392] INFO - CHECK FOR PRIMARY INDEXES
[2016-04-06 08:46:15,080] - [tuq_helper:399] INFO - DROP PRIMARY INDEX ON default USING GSI
[2016-04-06 08:46:15,080] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2016-04-06 08:46:15,083] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2016-04-06 08:46:15,096] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 12.056429ms
[2016-04-06 08:46:15,097] - [tuq_helper:75] INFO - RUN QUERY DROP PRIMARY INDEX ON default USING GSI
[2016-04-06 08:46:15,099] - [rest_client:2508] INFO - query params : statement=DROP+PRIMARY+INDEX+ON+default+USING+GSI
[2016-04-06 08:46:15,119] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 18.383495ms
[2016-04-06 08:46:15,362] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: rm -f /opt/couchbase/var/lib/couchbase/meta;rm -f /tmp/log_upr_client.sock
[2016-04-06 08:46:15,396] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:46:15,397] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: killall cbq-engine
[2016-04-06 08:46:15,439] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:46:15,439] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: killall tuqtng
[2016-04-06 08:46:15,494] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:46:15,495] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: killall indexer
[2016-04-06 08:46:15,549] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:46:15,550] - [basetestcase:297] INFO - ==============  basetestcase cleanup was started for test #8 test_multi_create_query_explain_drop_index ==============
[2016-04-06 08:46:15,579] - [bucket_helper:137] INFO - deleting existing buckets [u'default'] on 127.0.0.1
[2016-04-06 08:46:15,579] - [bucket_helper:139] INFO - remove bucket default ...
[2016-04-06 08:46:16,101] - [bucket_helper:153] INFO - deleted bucket : default from 127.0.0.1
[2016-04-06 08:46:16,101] - [bucket_helper:226] INFO - waiting for bucket deletion to complete....
[2016-04-06 08:46:16,103] - [rest_client:127] INFO - node 127.0.0.1 existing buckets : []
[2016-04-06 08:46:16,109] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:46:16,115] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:46:16,120] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:46:16,133] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9000
[2016-04-06 08:46:16,136] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9000 is running
[2016-04-06 08:46:16,138] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9001
[2016-04-06 08:46:16,141] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9001 is running
[2016-04-06 08:46:16,144] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9002
[2016-04-06 08:46:16,147] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9002 is running
[2016-04-06 08:46:16,150] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9003
[2016-04-06 08:46:16,152] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9003 is running
[2016-04-06 08:46:16,152] - [basetestcase:315] INFO - ==============  basetestcase cleanup was finished for test #8 test_multi_create_query_explain_drop_index ==============
ok

----------------------------------------------------------------------
Ran 1 test in 40.503s

OK
test_multi_create_query_explain_drop_index (2i.indexscans_2i.SecondaryIndexingScanTests) ... Cluster instance shutdown with force
summary so far suite 2i.indexscans_2i.SecondaryIndexingScanTests , pass 8 , fail 0
testrunner logs, diags and results are available under /opt/build/testrunner/logs/testrunner-16-Apr-06_08-41-08/test_8
Logs will be stored at /opt/build/testrunner/logs/testrunner-16-Apr-06_08-41-08/test_9

./testrunner -i b/resources/dev-4-nodes-xdcr_n1ql_2i.ini makefile=True -t 2i.indexscans_2i.SecondaryIndexingScanTests.test_multi_create_query_explain_drop_index,groups=simple:equals:no_orderby_groupby:range,dataset=default,doc-per-day=1,use_gsi_for_primary=True,use_gsi_for_secondary=True,doc_ops=True,create_ops_per=.5,delete_ops_per=.2,update_ops_per=.2,run_async=True,scan_consistency=request_plus

Test Input params:
{'doc-per-day': '1', 'use_gsi_for_primary': 'True', 'use_gsi_for_secondary': 'True', 'doc_ops': 'True', 'makefile': 'True', 'delete_ops_per': '.2', 'conf_file': 'conf/simple_2i_n1ql.conf', 'dataset': 'default', 'cluster_name': 'dev-4-nodes-xdcr_n1ql_2i', 'num_nodes': 4, 'update_ops_per': '.2', 'run_async': 'True', 'groups': 'simple:equals:no_orderby_groupby:range', 'case_number': 9, 'scan_consistency': 'request_plus', 'create_ops_per': '.5', 'logs_folder': '/opt/build/testrunner/logs/testrunner-16-Apr-06_08-41-08/test_9', 'spec': 'simple_2i_n1ql', 'ini': 'b/resources/dev-4-nodes-xdcr_n1ql_2i.ini'}
[2016-04-06 08:46:16,202] - [rest_client:1636] INFO - Node versions in cluster [u'4.5.35007-1459908767-rel-enterprise']
[2016-04-06 08:46:16,202] - [basetestcase:146] INFO - ==============  basetestcase setup was started for test #9 test_multi_create_query_explain_drop_index==============
[2016-04-06 08:46:16,216] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:46:16,221] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:46:16,226] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:46:16,231] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:46:16,240] - [basetestcase:365] INFO - sleep for 10 secs.  ...
[2016-04-06 08:46:26,253] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9000
[2016-04-06 08:46:26,256] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9000 is running
[2016-04-06 08:46:26,258] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9001
[2016-04-06 08:46:26,261] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9001 is running
[2016-04-06 08:46:26,264] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9002
[2016-04-06 08:46:26,266] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9002 is running
[2016-04-06 08:46:26,269] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9003
[2016-04-06 08:46:26,271] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9003 is running
[2016-04-06 08:46:26,272] - [basetestcase:167] INFO - initializing cluster
[2016-04-06 08:46:27,212] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:46:27,214] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:46:27,227] - [rest_client:789] INFO - /node/controller/setupServices params on 127.0.0.1: 9000:services=kv%2Cindex%2Cn1ql&password=asdasd&hostname=127.0.0.1%3A9000&user=Administrator
[2016-04-06 08:46:27,232] - [rest_client:757] ERROR - http://127.0.0.1:9000//node/controller/setupServices error 400 reason: unknown ["cannot change node services after cluster is provisioned"]
[2016-04-06 08:46:27,232] - [rest_client:794] INFO - This node is already provisioned with services, we do not consider this as failure for test case
[2016-04-06 08:46:27,232] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9000:username=Administrator&password=asdasd&port=9000
[2016-04-06 08:46:27,251] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:46:27,256] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:46:27,259] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9001:username=Administrator&password=asdasd&port=9001
[2016-04-06 08:46:27,276] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:46:27,279] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:46:27,283] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9002:username=Administrator&password=asdasd&port=9002
[2016-04-06 08:46:27,297] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:46:27,301] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:46:27,304] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9003:username=Administrator&password=asdasd&port=9003
[2016-04-06 08:46:27,315] - [basetestcase:187] INFO - done initializing cluster
[2016-04-06 08:46:28,329] - [rest_client:1772] INFO - http://127.0.0.1:9000/pools/default/buckets with param: bucketType=membase&evictionPolicy=valueOnly&threadsNumber=3&ramQuotaMB=8041&proxyPort=11211&authType=sasl&name=default&flushEnabled=1&replicaNumber=1&replicaIndex=1&saslPassword=None
[2016-04-06 08:46:28,334] - [rest_client:1794] INFO - 0.00 seconds to create bucket default
[2016-04-06 08:46:28,334] - [bucket_helper:305] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2016-04-06 08:46:29,405] - [data_helper:295] INFO - creating direct client 127.0.0.1:12000 default
[2016-04-06 08:46:29,440] - [data_helper:295] INFO - creating direct client 127.0.0.1:12000 default
[2016-04-06 08:46:29,464] - [task:288] INFO - bucket 'default' was created with per node RAM quota: 8041
[2016-04-06 08:46:29,490] - [basetestcase:257] INFO - ==============  basetestcase setup was finished for test #9 test_multi_create_query_explain_drop_index ==============
[2016-04-06 08:46:29,495] - [basetestcase:365] INFO - sleep for 10 secs.  ...
[2016-04-06 08:46:39,511] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:46:39,511] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:46:39,517] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:46:39,517] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:46:39,523] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:46:39,523] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:46:39,529] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:46:39,529] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:46:39,534] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:46:39,534] - [remote_util:168] INFO - connecting to 127.0.0.1 with username : Administrator password : asdasd ssh_key: 
[2016-04-06 08:46:39,534] - [remote_util:201] INFO - Connected to 127.0.0.1
[2016-04-06 08:46:40,457] - [basetestcase:2084] INFO - create 2016 to default documents...
[2016-04-06 08:46:40,475] - [data_helper:295] INFO - creating direct client 127.0.0.1:12000 default
[2016-04-06 08:46:42,659] - [basetestcase:2095] INFO - LOAD IS FINISHED
[2016-04-06 08:46:42,663] - [newtuq:82] INFO - {'remaining': {'start': 0, 'end': 1}, 'create': {'start': 1, 'end': 2.5}, 'update': {'start': 0, 'end': 0}, 'delete': {'start': 0, 'end': 0}}
[2016-04-06 08:46:44,250] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:46:44,250] - [newtuq:92] INFO - ip:127.0.0.1 port:9000 ssh_username:Administrator
number of buckets 1
[2016-04-06 08:46:44,251] - [tuq_helper:419] INFO - CREATE PRIMARY INDEX ON default  USING GSI
[2016-04-06 08:46:44,251] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2016-04-06 08:46:44,253] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2016-04-06 08:46:44,284] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 28.683533ms
[2016-04-06 08:46:44,285] - [tuq_helper:75] INFO - RUN QUERY CREATE PRIMARY INDEX ON default  USING GSI
[2016-04-06 08:46:44,293] - [rest_client:2508] INFO - query params : statement=CREATE+PRIMARY+INDEX+ON+default++USING+GSI
[2016-04-06 08:46:44,801] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 504.789003ms
[2016-04-06 08:46:44,801] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2016-04-06 08:46:44,804] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2016-04-06 08:46:44,826] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 19.941942ms
[2016-04-06 08:46:44,827] - [base_2i:48] INFO - {'in_between': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}, 'initial': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}, 'after': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}, 'before': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}}
[2016-04-06 08:46:44,841] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:46:44,846] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:46:45,484] - [tuq_helper:75] INFO - RUN QUERY CREATE INDEX employee223b239fd699488a8b766995a6d8b8fajob_title ON default(job_title) USING GSI  WITH {'nodes': ['127.0.0.1:9000'], 'defer_build': True}
[2016-04-06 08:46:45,487] - [rest_client:2508] INFO - query params : statement=CREATE+INDEX+employee223b239fd699488a8b766995a6d8b8fajob_title+ON+default%28job_title%29+USING+GSI++WITH+%7B%27nodes%27%3A+%5B%27127.0.0.1%3A9000%27%5D%2C+%27defer_build%27%3A+True%7D
[2016-04-06 08:46:45,520] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 31.646809ms
[2016-04-06 08:46:45,520] - [base_2i:102] INFO - BUILD INDEX on default(employee223b239fd699488a8b766995a6d8b8fajob_title) USING GSI
[2016-04-06 08:46:46,521] - [tuq_helper:75] INFO - RUN QUERY BUILD INDEX on default(employee223b239fd699488a8b766995a6d8b8fajob_title) USING GSI
[2016-04-06 08:46:46,525] - [rest_client:2508] INFO - query params : statement=BUILD+INDEX+on+default%28employee223b239fd699488a8b766995a6d8b8fajob_title%29+USING+GSI
[2016-04-06 08:46:46,536] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 9.534866ms
[2016-04-06 08:46:47,537] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee223b239fd699488a8b766995a6d8b8fajob_title'
[2016-04-06 08:46:47,541] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee223b239fd699488a8b766995a6d8b8fajob_title%27
[2016-04-06 08:46:47,568] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 25.591271ms
[2016-04-06 08:46:48,568] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee223b239fd699488a8b766995a6d8b8fajob_title'
[2016-04-06 08:46:48,571] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee223b239fd699488a8b766995a6d8b8fajob_title%27
[2016-04-06 08:46:48,586] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 13.045522ms
[2016-04-06 08:46:49,074] - [basetestcase:2084] INFO - create 3024.0 to default documents...
[2016-04-06 08:46:49,095] - [data_helper:295] INFO - creating direct client 127.0.0.1:12000 default
[2016-04-06 08:46:53,008] - [basetestcase:2095] INFO - LOAD IS FINISHED
[2016-04-06 08:46:53,455] - [basetestcase:2084] INFO - delete 0 to default documents...
[2016-04-06 08:46:53,472] - [data_helper:295] INFO - creating direct client 127.0.0.1:12000 default
[2016-04-06 08:46:53,596] - [basetestcase:2095] INFO - LOAD IS FINISHED
[2016-04-06 08:46:54,068] - [basetestcase:2084] INFO - update 0 to default documents...
[2016-04-06 08:46:54,088] - [data_helper:295] INFO - creating direct client 127.0.0.1:12000 default
[2016-04-06 08:46:54,601] - [basetestcase:2095] INFO - LOAD IS FINISHED
[2016-04-06 08:46:55,606] - [task:2250] INFO -  <<<<< START Executing Query EXPLAIN SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2016-04-06 08:46:55,606] - [tuq_helper:75] INFO - RUN QUERY EXPLAIN SELECT * FROM default WHERE  job_title = "Sales" 
[2016-04-06 08:46:55,609] - [rest_client:2508] INFO - query params : statement=EXPLAIN+SELECT+%2A+FROM+default+WHERE++job_title+%3D+%22Sales%22+
[2016-04-06 08:46:55,614] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 3.067232ms
[2016-04-06 08:46:55,614] - [task:2259] INFO - {u'status': u'success', u'metrics': {u'elapsedTime': u'3.067232ms', u'executionTime': u'2.998413ms', u'resultSize': 2409, u'resultCount': 1}, u'results': [{u'text': u'SELECT * FROM default WHERE  job_title = "Sales"', u'plan': {u'#operator': u'Sequence', u'~children': [{u'index': u'employee223b239fd699488a8b766995a6d8b8fajob_title', u'namespace': u'default', u'keyspace': u'default', u'#operator': u'IndexScan', u'spans': [{u'Range': {u'High': [u'"Sales"'], u'Low': [u'"Sales"'], u'Inclusion': 3}}], u'using': u'gsi', u'index_id': u'ca8f1924a8afa9df'}, {u'#operator': u'Parallel', u'~child': {u'#operator': u'Sequence', u'~children': [{u'keyspace': u'default', u'#operator': u'Fetch', u'namespace': u'default'}, {u'#operator': u'Filter', u'condition': u'((`default`.`job_title`) = "Sales")'}, {u'#operator': u'InitialProject', u'result_terms': [{u'expr': u'self', u'star': True}]}, {u'#operator': u'FinalProject'}]}}]}}], u'requestID': u'fd02c1b9-c053-4624-a1d3-00270066e4e6', u'signature': u'json'}
[2016-04-06 08:46:55,614] - [task:2260] INFO -  <<<<< Done Executing Query EXPLAIN SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2016-04-06 08:46:55,614] - [task:2289] INFO -  <<<<< Done VERIFYING Query EXPLAIN SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2016-04-06 08:46:55,614] - [base_2i:294] INFO - Query : SELECT * FROM default WHERE  job_title = "Sales" 
[2016-04-06 08:46:55,615] - [tuq_generators:66] INFO - FROM clause ===== is default
[2016-04-06 08:46:55,615] - [tuq_generators:68] INFO - WHERE clause ===== is   doc["job_title"] == "Sales" 
[2016-04-06 08:46:55,615] - [tuq_generators:70] INFO - UNNEST clause ===== is None
[2016-04-06 08:46:55,615] - [tuq_generators:72] INFO - SELECT clause ===== is {"*" : doc,}
[2016-04-06 08:46:56,615] - [task:2250] INFO -  <<<<< START Executing Query SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2016-04-06 08:46:56,615] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM default WHERE  job_title = "Sales" 
[2016-04-06 08:46:56,618] - [rest_client:2508] INFO - query params : scan_consistency=request_plus&statement=SELECT+%2A+FROM+default+WHERE++job_title+%3D+%22Sales%22+
[2016-04-06 08:46:57,130] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 470.573419ms
[2016-04-06 08:46:57,184] - [tuq_helper:108] INFO -  Analyzing Actual Result
[2016-04-06 08:46:57,186] - [tuq_helper:110] INFO -  Analyzing Expected Result
[2016-04-06 08:46:57,198] - [task:2260] INFO -  <<<<< Done Executing Query SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2016-04-06 08:46:57,198] - [task:2289] INFO -  <<<<< Done VERIFYING Query SELECT * FROM default WHERE  job_title = "Sales"  >>>>>>
[2016-04-06 08:46:58,200] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee223b239fd699488a8b766995a6d8b8fajob_title'
[2016-04-06 08:46:58,203] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee223b239fd699488a8b766995a6d8b8fajob_title%27
[2016-04-06 08:46:58,218] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 13.308217ms
[2016-04-06 08:46:58,218] - [tuq_helper:75] INFO - RUN QUERY DROP INDEX default.employee223b239fd699488a8b766995a6d8b8fajob_title USING GSI
[2016-04-06 08:46:58,221] - [rest_client:2508] INFO - query params : statement=DROP+INDEX+default.employee223b239fd699488a8b766995a6d8b8fajob_title+USING+GSI
[2016-04-06 08:46:58,242] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 19.203716ms
[2016-04-06 08:46:58,242] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee223b239fd699488a8b766995a6d8b8fajob_title'
[2016-04-06 08:46:58,253] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee223b239fd699488a8b766995a6d8b8fajob_title%27
[2016-04-06 08:46:58,272] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 17.273983ms
[2016-04-06 08:46:58,280] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:46:58,281] - [remote_util:168] INFO - connecting to 127.0.0.1 with username : Administrator password : asdasd ssh_key: 
[2016-04-06 08:46:58,281] - [remote_util:201] INFO - Connected to 127.0.0.1
[2016-04-06 08:46:58,288] - [rest_client:1222] INFO - /diag/eval status on 127.0.0.1:9000: True content: "/opt/build/ns_server/logs/n_0" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))).
[2016-04-06 08:46:58,584] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: zgrep "panic" "/opt/build/ns_server/logs/n_0"/indexer.log* | wc -l
[2016-04-06 08:46:58,630] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:46:58,636] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:46:58,637] - [tuq_helper:392] INFO - CHECK FOR PRIMARY INDEXES
[2016-04-06 08:46:58,637] - [tuq_helper:399] INFO - DROP PRIMARY INDEX ON default USING GSI
[2016-04-06 08:46:58,637] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2016-04-06 08:46:58,640] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2016-04-06 08:46:58,661] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 20.185363ms
[2016-04-06 08:46:58,662] - [tuq_helper:75] INFO - RUN QUERY DROP PRIMARY INDEX ON default USING GSI
[2016-04-06 08:46:58,665] - [rest_client:2508] INFO - query params : statement=DROP+PRIMARY+INDEX+ON+default+USING+GSI
[2016-04-06 08:46:58,687] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 20.388369ms
[2016-04-06 08:46:58,974] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: rm -f /opt/couchbase/var/lib/couchbase/meta;rm -f /tmp/log_upr_client.sock
[2016-04-06 08:46:59,015] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:46:59,016] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: killall cbq-engine
[2016-04-06 08:46:59,063] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:46:59,063] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: killall tuqtng
[2016-04-06 08:46:59,128] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:46:59,128] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: killall indexer
[2016-04-06 08:46:59,184] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:46:59,185] - [basetestcase:297] INFO - ==============  basetestcase cleanup was started for test #9 test_multi_create_query_explain_drop_index ==============
[2016-04-06 08:46:59,213] - [bucket_helper:137] INFO - deleting existing buckets [u'default'] on 127.0.0.1
[2016-04-06 08:46:59,213] - [bucket_helper:139] INFO - remove bucket default ...
[2016-04-06 08:47:00,642] - [bucket_helper:153] INFO - deleted bucket : default from 127.0.0.1
[2016-04-06 08:47:00,642] - [bucket_helper:226] INFO - waiting for bucket deletion to complete....
[2016-04-06 08:47:00,648] - [rest_client:127] INFO - node 127.0.0.1 existing buckets : []
[2016-04-06 08:47:00,653] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:47:00,658] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:47:00,665] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:47:00,677] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9000
[2016-04-06 08:47:00,679] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9000 is running
[2016-04-06 08:47:00,682] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9001
[2016-04-06 08:47:00,684] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9001 is running
[2016-04-06 08:47:00,688] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9002
[2016-04-06 08:47:00,690] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9002 is running
[2016-04-06 08:47:00,693] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9003
[2016-04-06 08:47:00,696] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9003 is running
[2016-04-06 08:47:00,696] - [basetestcase:315] INFO - ==============  basetestcase cleanup was finished for test #9 test_multi_create_query_explain_drop_index ==============
ok

----------------------------------------------------------------------
Ran 1 test in 44.507s

OK
test_multi_create_drop_index (2i.indexcreatedrop_2i.SecondaryIndexingCreateDropTests) ... Cluster instance shutdown with force
summary so far suite 2i.indexscans_2i.SecondaryIndexingScanTests , pass 9 , fail 0
testrunner logs, diags and results are available under /opt/build/testrunner/logs/testrunner-16-Apr-06_08-41-08/test_9
Logs will be stored at /opt/build/testrunner/logs/testrunner-16-Apr-06_08-41-08/test_10

./testrunner -i b/resources/dev-4-nodes-xdcr_n1ql_2i.ini makefile=True -t 2i.indexcreatedrop_2i.SecondaryIndexingCreateDropTests.test_multi_create_drop_index,groups=simple,dataset=default,doc-per-day=1,cbq_version=sherlock,skip_build_tuq=True

Test Input params:
{'doc-per-day': '1', 'logs_folder': '/opt/build/testrunner/logs/testrunner-16-Apr-06_08-41-08/test_10', 'makefile': 'True', 'skip_build_tuq': 'True', 'conf_file': 'conf/simple_2i_n1ql.conf', 'dataset': 'default', 'cluster_name': 'dev-4-nodes-xdcr_n1ql_2i', 'ini': 'b/resources/dev-4-nodes-xdcr_n1ql_2i.ini', 'groups': 'simple', 'case_number': 10, 'num_nodes': 4, 'spec': 'simple_2i_n1ql', 'cbq_version': 'sherlock'}
[2016-04-06 08:47:00,767] - [rest_client:1636] INFO - Node versions in cluster [u'4.5.35007-1459908767-rel-enterprise']
[2016-04-06 08:47:00,767] - [basetestcase:146] INFO - ==============  basetestcase setup was started for test #10 test_multi_create_drop_index==============
[2016-04-06 08:47:00,788] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:47:00,793] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:47:00,797] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:47:00,802] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:47:00,812] - [basetestcase:365] INFO - sleep for 10 secs.  ...
[2016-04-06 08:47:10,819] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9000
[2016-04-06 08:47:10,821] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9000 is running
[2016-04-06 08:47:10,824] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9001
[2016-04-06 08:47:10,827] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9001 is running
[2016-04-06 08:47:10,829] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9002
[2016-04-06 08:47:10,832] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9002 is running
[2016-04-06 08:47:10,835] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9003
[2016-04-06 08:47:10,837] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9003 is running
[2016-04-06 08:47:10,837] - [basetestcase:167] INFO - initializing cluster
[2016-04-06 08:47:11,778] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:47:11,781] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:47:11,807] - [rest_client:789] INFO - /node/controller/setupServices params on 127.0.0.1: 9000:services=kv%2Cindex%2Cn1ql&password=asdasd&hostname=127.0.0.1%3A9000&user=Administrator
[2016-04-06 08:47:11,809] - [rest_client:757] ERROR - http://127.0.0.1:9000//node/controller/setupServices error 400 reason: unknown ["cannot change node services after cluster is provisioned"]
[2016-04-06 08:47:11,809] - [rest_client:794] INFO - This node is already provisioned with services, we do not consider this as failure for test case
[2016-04-06 08:47:11,809] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9000:username=Administrator&password=asdasd&port=9000
[2016-04-06 08:47:11,827] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:47:11,830] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:47:11,834] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9001:username=Administrator&password=asdasd&port=9001
[2016-04-06 08:47:11,850] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:47:11,853] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:47:11,857] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9002:username=Administrator&password=asdasd&port=9002
[2016-04-06 08:47:11,871] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:47:11,875] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:47:11,880] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9003:username=Administrator&password=asdasd&port=9003
[2016-04-06 08:47:11,890] - [basetestcase:187] INFO - done initializing cluster
[2016-04-06 08:47:12,903] - [rest_client:1772] INFO - http://127.0.0.1:9000/pools/default/buckets with param: bucketType=membase&evictionPolicy=valueOnly&threadsNumber=3&ramQuotaMB=8041&proxyPort=11211&authType=sasl&name=default&flushEnabled=1&replicaNumber=1&replicaIndex=1&saslPassword=None
[2016-04-06 08:47:12,912] - [rest_client:1794] INFO - 0.01 seconds to create bucket default
[2016-04-06 08:47:12,913] - [bucket_helper:305] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2016-04-06 08:47:13,995] - [data_helper:295] INFO - creating direct client 127.0.0.1:12000 default
[2016-04-06 08:47:14,023] - [data_helper:295] INFO - creating direct client 127.0.0.1:12000 default
[2016-04-06 08:47:14,053] - [task:288] INFO - bucket 'default' was created with per node RAM quota: 8041
[2016-04-06 08:47:14,065] - [basetestcase:257] INFO - ==============  basetestcase setup was finished for test #10 test_multi_create_drop_index ==============
[2016-04-06 08:47:14,069] - [basetestcase:365] INFO - sleep for 10 secs.  ...
[2016-04-06 08:47:24,076] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:47:24,077] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:47:24,083] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:47:24,083] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:47:24,088] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:47:24,089] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:47:24,096] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:47:24,096] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:47:24,101] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:47:24,101] - [remote_util:168] INFO - connecting to 127.0.0.1 with username : Administrator password : asdasd ssh_key: 
[2016-04-06 08:47:24,102] - [remote_util:201] INFO - Connected to 127.0.0.1
[2016-04-06 08:47:25,021] - [basetestcase:2084] INFO - create 2016 to default documents...
[2016-04-06 08:47:25,041] - [data_helper:295] INFO - creating direct client 127.0.0.1:12000 default
[2016-04-06 08:47:26,166] - [basetestcase:2095] INFO - LOAD IS FINISHED
[2016-04-06 08:47:26,177] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:47:26,177] - [newtuq:92] INFO - ip:127.0.0.1 port:9000 ssh_username:Administrator
number of buckets 1
[2016-04-06 08:47:26,177] - [tuq_helper:419] INFO - CREATE PRIMARY INDEX ON default  USING VIEW 
[2016-04-06 08:47:26,178] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2016-04-06 08:47:26,181] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2016-04-06 08:47:26,217] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 34.466919ms
[2016-04-06 08:47:26,217] - [tuq_helper:75] INFO - RUN QUERY CREATE PRIMARY INDEX ON default  USING VIEW 
[2016-04-06 08:47:26,220] - [rest_client:2508] INFO - query params : statement=CREATE+PRIMARY+INDEX+ON+default++USING+VIEW+
[2016-04-06 08:47:31,262] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 5.039923642s
[2016-04-06 08:47:31,262] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2016-04-06 08:47:31,265] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2016-04-06 08:47:31,285] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 18.01633ms
[2016-04-06 08:47:31,286] - [base_2i:48] INFO - {'in_between': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}, 'initial': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}, 'after': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}, 'before': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}}
[2016-04-06 08:47:31,300] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:47:31,300] - [base_2i:134] INFO - []
[2016-04-06 08:47:31,301] - [basetestcase:365] INFO - sleep for 3 secs.  ...
[2016-04-06 08:47:32,075] - [tuq_helper:75] INFO - RUN QUERY CREATE INDEX employeed08c6ee84dc849fb9f7f754a30583b90job_title ON default(job_title) USING GSI  WITH {'defer_build': True}
[2016-04-06 08:47:32,078] - [rest_client:2508] INFO - query params : statement=CREATE+INDEX+employeed08c6ee84dc849fb9f7f754a30583b90job_title+ON+default%28job_title%29+USING+GSI++WITH+%7B%27defer_build%27%3A+True%7D
[2016-04-06 08:47:32,134] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 53.715809ms
[2016-04-06 08:47:34,304] - [basetestcase:365] INFO - sleep for 3 secs.  ...
[2016-04-06 08:47:35,138] - [tuq_helper:75] INFO - RUN QUERY CREATE INDEX employeed08c6ee84dc849fb9f7f754a30583b90join_yr ON default(join_yr) USING GSI  WITH {'defer_build': True}
[2016-04-06 08:47:35,141] - [rest_client:2508] INFO - query params : statement=CREATE+INDEX+employeed08c6ee84dc849fb9f7f754a30583b90join_yr+ON+default%28join_yr%29+USING+GSI++WITH+%7B%27defer_build%27%3A+True%7D
[2016-04-06 08:47:35,185] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 42.743057ms
[2016-04-06 08:47:37,307] - [base_2i:102] INFO - BUILD INDEX on default(employeed08c6ee84dc849fb9f7f754a30583b90job_title,employeed08c6ee84dc849fb9f7f754a30583b90join_yr) USING GSI
[2016-04-06 08:47:38,187] - [tuq_helper:75] INFO - RUN QUERY BUILD INDEX on default(employeed08c6ee84dc849fb9f7f754a30583b90job_title,employeed08c6ee84dc849fb9f7f754a30583b90join_yr) USING GSI
[2016-04-06 08:47:38,190] - [rest_client:2508] INFO - query params : statement=BUILD+INDEX+on+default%28employeed08c6ee84dc849fb9f7f754a30583b90job_title%2Cemployeed08c6ee84dc849fb9f7f754a30583b90join_yr%29+USING+GSI
[2016-04-06 08:47:38,203] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 10.772201ms
[2016-04-06 08:47:39,204] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employeed08c6ee84dc849fb9f7f754a30583b90job_title'
[2016-04-06 08:47:39,207] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employeed08c6ee84dc849fb9f7f754a30583b90job_title%27
[2016-04-06 08:47:39,226] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 17.521752ms
[2016-04-06 08:47:39,227] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employeed08c6ee84dc849fb9f7f754a30583b90join_yr'
[2016-04-06 08:47:39,229] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employeed08c6ee84dc849fb9f7f754a30583b90join_yr%27
[2016-04-06 08:47:39,247] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 16.142373ms
[2016-04-06 08:47:39,247] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employeed08c6ee84dc849fb9f7f754a30583b90job_title'
[2016-04-06 08:47:39,250] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employeed08c6ee84dc849fb9f7f754a30583b90job_title%27
[2016-04-06 08:47:39,274] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 22.769575ms
[2016-04-06 08:47:39,274] - [tuq_helper:75] INFO - RUN QUERY DROP INDEX default.employeed08c6ee84dc849fb9f7f754a30583b90job_title USING GSI
[2016-04-06 08:47:39,277] - [rest_client:2508] INFO - query params : statement=DROP+INDEX+default.employeed08c6ee84dc849fb9f7f754a30583b90job_title+USING+GSI
[2016-04-06 08:47:39,298] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 19.448844ms
[2016-04-06 08:47:39,298] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employeed08c6ee84dc849fb9f7f754a30583b90join_yr'
[2016-04-06 08:47:39,302] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employeed08c6ee84dc849fb9f7f754a30583b90join_yr%27
[2016-04-06 08:47:39,324] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 19.687611ms
[2016-04-06 08:47:39,324] - [tuq_helper:75] INFO - RUN QUERY DROP INDEX default.employeed08c6ee84dc849fb9f7f754a30583b90join_yr USING GSI
[2016-04-06 08:47:39,327] - [rest_client:2508] INFO - query params : statement=DROP+INDEX+default.employeed08c6ee84dc849fb9f7f754a30583b90join_yr+USING+GSI
[2016-04-06 08:47:39,350] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 21.305722ms
[2016-04-06 08:47:39,350] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employeed08c6ee84dc849fb9f7f754a30583b90job_title'
[2016-04-06 08:47:39,356] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employeed08c6ee84dc849fb9f7f754a30583b90job_title%27
[2016-04-06 08:47:39,381] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 22.988452ms
[2016-04-06 08:47:39,381] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employeed08c6ee84dc849fb9f7f754a30583b90join_yr'
[2016-04-06 08:47:39,384] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employeed08c6ee84dc849fb9f7f754a30583b90join_yr%27
[2016-04-06 08:47:39,399] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 14.454405ms
[2016-04-06 08:47:39,405] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:47:39,405] - [remote_util:168] INFO - connecting to 127.0.0.1 with username : Administrator password : asdasd ssh_key: 
[2016-04-06 08:47:39,405] - [remote_util:201] INFO - Connected to 127.0.0.1
[2016-04-06 08:47:39,410] - [rest_client:1222] INFO - /diag/eval status on 127.0.0.1:9000: True content: "/opt/build/ns_server/logs/n_0" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))).
[2016-04-06 08:47:39,596] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: zgrep "panic" "/opt/build/ns_server/logs/n_0"/indexer.log* | wc -l
[2016-04-06 08:47:39,624] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:47:39,631] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:47:39,631] - [tuq_helper:392] INFO - CHECK FOR PRIMARY INDEXES
[2016-04-06 08:47:39,631] - [tuq_helper:399] INFO - DROP PRIMARY INDEX ON default USING VIEW 
[2016-04-06 08:47:39,631] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2016-04-06 08:47:39,634] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2016-04-06 08:47:39,656] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 20.131743ms
[2016-04-06 08:47:39,656] - [tuq_helper:75] INFO - RUN QUERY DROP PRIMARY INDEX ON default USING VIEW 
[2016-04-06 08:47:39,658] - [rest_client:2508] INFO - query params : statement=DROP+PRIMARY+INDEX+ON+default+USING+VIEW+
[2016-04-06 08:47:39,665] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 4.648902ms
[2016-04-06 08:47:39,840] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: rm -f /opt/couchbase/var/lib/couchbase/meta;rm -f /tmp/log_upr_client.sock
[2016-04-06 08:47:39,866] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:47:39,867] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: killall cbq-engine
[2016-04-06 08:47:39,897] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:47:39,898] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: killall tuqtng
[2016-04-06 08:47:39,937] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:47:39,938] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: killall indexer
[2016-04-06 08:47:39,973] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:47:39,973] - [basetestcase:297] INFO - ==============  basetestcase cleanup was started for test #10 test_multi_create_drop_index ==============
[2016-04-06 08:47:40,003] - [bucket_helper:137] INFO - deleting existing buckets [u'default'] on 127.0.0.1
[2016-04-06 08:47:40,003] - [bucket_helper:139] INFO - remove bucket default ...
[2016-04-06 08:47:41,188] - [bucket_helper:153] INFO - deleted bucket : default from 127.0.0.1
[2016-04-06 08:47:41,189] - [bucket_helper:226] INFO - waiting for bucket deletion to complete....
[2016-04-06 08:47:41,191] - [rest_client:127] INFO - node 127.0.0.1 existing buckets : []
[2016-04-06 08:47:41,197] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:47:41,203] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:47:41,211] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:47:41,230] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9000
[2016-04-06 08:47:41,236] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9000 is running
[2016-04-06 08:47:41,240] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9001
[2016-04-06 08:47:41,243] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9001 is running
[2016-04-06 08:47:41,246] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9002
[2016-04-06 08:47:41,250] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9002 is running
[2016-04-06 08:47:41,253] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9003
[2016-04-06 08:47:41,256] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9003 is running
[2016-04-06 08:47:41,256] - [basetestcase:315] INFO - ==============  basetestcase cleanup was finished for test #10 test_multi_create_drop_index ==============
ok

----------------------------------------------------------------------
Ran 1 test in 40.508s

OK
test_multi_create_drop_index (2i.indexcreatedrop_2i.SecondaryIndexingCreateDropTests) ... Cluster instance shutdown with force
summary so far suite 2i.indexscans_2i.SecondaryIndexingScanTests , pass 9 , fail 0
summary so far suite 2i.indexcreatedrop_2i.SecondaryIndexingCreateDropTests , pass 1 , fail 0
testrunner logs, diags and results are available under /opt/build/testrunner/logs/testrunner-16-Apr-06_08-41-08/test_10
Logs will be stored at /opt/build/testrunner/logs/testrunner-16-Apr-06_08-41-08/test_11

./testrunner -i b/resources/dev-4-nodes-xdcr_n1ql_2i.ini makefile=True -t 2i.indexcreatedrop_2i.SecondaryIndexingCreateDropTests.test_multi_create_drop_index,groups=composite,dataset=default,doc-per-day=1,cbq_version=sherlock,skip_build_tuq=True

Test Input params:
{'doc-per-day': '1', 'logs_folder': '/opt/build/testrunner/logs/testrunner-16-Apr-06_08-41-08/test_11', 'makefile': 'True', 'skip_build_tuq': 'True', 'conf_file': 'conf/simple_2i_n1ql.conf', 'dataset': 'default', 'cluster_name': 'dev-4-nodes-xdcr_n1ql_2i', 'ini': 'b/resources/dev-4-nodes-xdcr_n1ql_2i.ini', 'groups': 'composite', 'case_number': 11, 'num_nodes': 4, 'spec': 'simple_2i_n1ql', 'cbq_version': 'sherlock'}
[2016-04-06 08:47:41,297] - [rest_client:1636] INFO - Node versions in cluster [u'4.5.35007-1459908767-rel-enterprise']
[2016-04-06 08:47:41,297] - [basetestcase:146] INFO - ==============  basetestcase setup was started for test #11 test_multi_create_drop_index==============
[2016-04-06 08:47:41,316] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:47:41,321] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:47:41,325] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:47:41,330] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:47:41,340] - [basetestcase:365] INFO - sleep for 10 secs.  ...
[2016-04-06 08:47:51,345] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9000
[2016-04-06 08:47:51,347] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9000 is running
[2016-04-06 08:47:51,351] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9001
[2016-04-06 08:47:51,356] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9001 is running
[2016-04-06 08:47:51,359] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9002
[2016-04-06 08:47:51,362] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9002 is running
[2016-04-06 08:47:51,365] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9003
[2016-04-06 08:47:51,367] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9003 is running
[2016-04-06 08:47:51,367] - [basetestcase:167] INFO - initializing cluster
[2016-04-06 08:47:52,309] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:47:52,312] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:47:52,316] - [rest_client:789] INFO - /node/controller/setupServices params on 127.0.0.1: 9000:services=kv%2Cindex%2Cn1ql&password=asdasd&hostname=127.0.0.1%3A9000&user=Administrator
[2016-04-06 08:47:52,332] - [rest_client:757] ERROR - http://127.0.0.1:9000//node/controller/setupServices error 400 reason: unknown ["cannot change node services after cluster is provisioned"]
[2016-04-06 08:47:52,332] - [rest_client:794] INFO - This node is already provisioned with services, we do not consider this as failure for test case
[2016-04-06 08:47:52,332] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9000:username=Administrator&password=asdasd&port=9000
[2016-04-06 08:47:52,357] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:47:52,360] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:47:52,365] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9001:username=Administrator&password=asdasd&port=9001
[2016-04-06 08:47:52,384] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:47:52,388] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:47:52,392] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9002:username=Administrator&password=asdasd&port=9002
[2016-04-06 08:47:52,409] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:47:52,413] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:47:52,418] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9003:username=Administrator&password=asdasd&port=9003
[2016-04-06 08:47:52,435] - [basetestcase:187] INFO - done initializing cluster
[2016-04-06 08:47:53,444] - [rest_client:1772] INFO - http://127.0.0.1:9000/pools/default/buckets with param: bucketType=membase&evictionPolicy=valueOnly&threadsNumber=3&ramQuotaMB=8041&proxyPort=11211&authType=sasl&name=default&flushEnabled=1&replicaNumber=1&replicaIndex=1&saslPassword=None
[2016-04-06 08:47:53,448] - [rest_client:1794] INFO - 0.00 seconds to create bucket default
[2016-04-06 08:47:53,448] - [bucket_helper:305] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2016-04-06 08:47:54,514] - [data_helper:295] INFO - creating direct client 127.0.0.1:12000 default
[2016-04-06 08:47:54,554] - [data_helper:295] INFO - creating direct client 127.0.0.1:12000 default
[2016-04-06 08:47:54,578] - [task:288] INFO - bucket 'default' was created with per node RAM quota: 8041
[2016-04-06 08:47:54,614] - [basetestcase:257] INFO - ==============  basetestcase setup was finished for test #11 test_multi_create_drop_index ==============
[2016-04-06 08:47:54,619] - [basetestcase:365] INFO - sleep for 10 secs.  ...
[2016-04-06 08:48:04,631] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:48:04,631] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:48:04,637] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:48:04,637] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:48:04,643] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:48:04,643] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:48:04,649] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:48:04,649] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:48:04,655] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:48:04,655] - [remote_util:168] INFO - connecting to 127.0.0.1 with username : Administrator password : asdasd ssh_key: 
[2016-04-06 08:48:04,655] - [remote_util:201] INFO - Connected to 127.0.0.1
[2016-04-06 08:48:05,586] - [basetestcase:2084] INFO - create 2016 to default documents...
[2016-04-06 08:48:05,603] - [data_helper:295] INFO - creating direct client 127.0.0.1:12000 default
[2016-04-06 08:48:07,703] - [basetestcase:2095] INFO - LOAD IS FINISHED
[2016-04-06 08:48:07,714] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:48:07,714] - [newtuq:92] INFO - ip:127.0.0.1 port:9000 ssh_username:Administrator
number of buckets 1
[2016-04-06 08:48:07,714] - [tuq_helper:419] INFO - CREATE PRIMARY INDEX ON default  USING VIEW 
[2016-04-06 08:48:07,714] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2016-04-06 08:48:07,721] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2016-04-06 08:48:07,754] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 31.814186ms
[2016-04-06 08:48:07,754] - [tuq_helper:75] INFO - RUN QUERY CREATE PRIMARY INDEX ON default  USING VIEW 
[2016-04-06 08:48:07,758] - [rest_client:2508] INFO - query params : statement=CREATE+PRIMARY+INDEX+ON+default++USING+VIEW+
[2016-04-06 08:48:08,185] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 425.056218ms
[2016-04-06 08:48:08,185] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2016-04-06 08:48:08,188] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2016-04-06 08:48:08,206] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 16.545767ms
[2016-04-06 08:48:08,207] - [base_2i:48] INFO - {'in_between': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}, 'initial': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}, 'after': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}, 'before': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}}
[2016-04-06 08:48:08,217] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:48:08,217] - [base_2i:134] INFO - []
[2016-04-06 08:48:08,218] - [basetestcase:365] INFO - sleep for 3 secs.  ...
[2016-04-06 08:48:08,596] - [tuq_helper:75] INFO - RUN QUERY CREATE INDEX employee0be1ca12f35e428992591f68cd84b199job_title_join_yr ON default(join_yr,job_title) USING GSI  WITH {'defer_build': True}
[2016-04-06 08:48:08,599] - [rest_client:2508] INFO - query params : statement=CREATE+INDEX+employee0be1ca12f35e428992591f68cd84b199job_title_join_yr+ON+default%28join_yr%2Cjob_title%29+USING+GSI++WITH+%7B%27defer_build%27%3A+True%7D
[2016-04-06 08:48:08,664] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 63.692987ms
[2016-04-06 08:48:11,221] - [base_2i:102] INFO - BUILD INDEX on default(employee0be1ca12f35e428992591f68cd84b199job_title_join_yr) USING GSI
[2016-04-06 08:48:11,667] - [tuq_helper:75] INFO - RUN QUERY BUILD INDEX on default(employee0be1ca12f35e428992591f68cd84b199job_title_join_yr) USING GSI
[2016-04-06 08:48:11,670] - [rest_client:2508] INFO - query params : statement=BUILD+INDEX+on+default%28employee0be1ca12f35e428992591f68cd84b199job_title_join_yr%29+USING+GSI
[2016-04-06 08:48:11,689] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 17.748581ms
[2016-04-06 08:48:12,691] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee0be1ca12f35e428992591f68cd84b199job_title_join_yr'
[2016-04-06 08:48:12,694] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee0be1ca12f35e428992591f68cd84b199job_title_join_yr%27
[2016-04-06 08:48:12,717] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 21.695536ms
[2016-04-06 08:48:13,719] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee0be1ca12f35e428992591f68cd84b199job_title_join_yr'
[2016-04-06 08:48:13,722] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee0be1ca12f35e428992591f68cd84b199job_title_join_yr%27
[2016-04-06 08:48:13,744] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 20.044627ms
[2016-04-06 08:48:13,744] - [tuq_helper:75] INFO - RUN QUERY DROP INDEX default.employee0be1ca12f35e428992591f68cd84b199job_title_join_yr USING GSI
[2016-04-06 08:48:13,747] - [rest_client:2508] INFO - query params : statement=DROP+INDEX+default.employee0be1ca12f35e428992591f68cd84b199job_title_join_yr+USING+GSI
[2016-04-06 08:48:13,771] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 22.513121ms
[2016-04-06 08:48:13,771] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee0be1ca12f35e428992591f68cd84b199job_title_join_yr'
[2016-04-06 08:48:13,781] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee0be1ca12f35e428992591f68cd84b199job_title_join_yr%27
[2016-04-06 08:48:13,799] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 16.300061ms
[2016-04-06 08:48:13,806] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:48:13,807] - [remote_util:168] INFO - connecting to 127.0.0.1 with username : Administrator password : asdasd ssh_key: 
[2016-04-06 08:48:13,807] - [remote_util:201] INFO - Connected to 127.0.0.1
[2016-04-06 08:48:13,812] - [rest_client:1222] INFO - /diag/eval status on 127.0.0.1:9000: True content: "/opt/build/ns_server/logs/n_0" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))).
[2016-04-06 08:48:13,997] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: zgrep "panic" "/opt/build/ns_server/logs/n_0"/indexer.log* | wc -l
[2016-04-06 08:48:14,028] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:48:14,035] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:48:14,036] - [tuq_helper:392] INFO - CHECK FOR PRIMARY INDEXES
[2016-04-06 08:48:14,036] - [tuq_helper:399] INFO - DROP PRIMARY INDEX ON default USING VIEW 
[2016-04-06 08:48:14,036] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2016-04-06 08:48:14,039] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2016-04-06 08:48:14,058] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 17.082586ms
[2016-04-06 08:48:14,058] - [tuq_helper:75] INFO - RUN QUERY DROP PRIMARY INDEX ON default USING VIEW 
[2016-04-06 08:48:14,061] - [rest_client:2508] INFO - query params : statement=DROP+PRIMARY+INDEX+ON+default+USING+VIEW+
[2016-04-06 08:48:14,068] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 5.425596ms
[2016-04-06 08:48:14,243] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: rm -f /opt/couchbase/var/lib/couchbase/meta;rm -f /tmp/log_upr_client.sock
[2016-04-06 08:48:14,270] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:48:14,271] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: killall cbq-engine
[2016-04-06 08:48:14,300] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:48:14,301] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: killall tuqtng
[2016-04-06 08:48:14,342] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:48:14,343] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: killall indexer
[2016-04-06 08:48:14,387] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:48:14,387] - [basetestcase:297] INFO - ==============  basetestcase cleanup was started for test #11 test_multi_create_drop_index ==============
[2016-04-06 08:48:14,410] - [bucket_helper:137] INFO - deleting existing buckets [u'default'] on 127.0.0.1
[2016-04-06 08:48:14,410] - [bucket_helper:139] INFO - remove bucket default ...
[2016-04-06 08:48:15,713] - [bucket_helper:153] INFO - deleted bucket : default from 127.0.0.1
[2016-04-06 08:48:15,713] - [bucket_helper:226] INFO - waiting for bucket deletion to complete....
[2016-04-06 08:48:15,720] - [rest_client:127] INFO - node 127.0.0.1 existing buckets : []
[2016-04-06 08:48:15,726] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:48:15,731] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:48:15,736] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:48:15,749] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9000
[2016-04-06 08:48:15,762] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9000 is running
[2016-04-06 08:48:15,765] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9001
[2016-04-06 08:48:15,767] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9001 is running
[2016-04-06 08:48:15,770] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9002
[2016-04-06 08:48:15,773] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9002 is running
[2016-04-06 08:48:15,778] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9003
[2016-04-06 08:48:15,785] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9003 is running
[2016-04-06 08:48:15,785] - [basetestcase:315] INFO - ==============  basetestcase cleanup was finished for test #11 test_multi_create_drop_index ==============
ok

----------------------------------------------------------------------
Ran 1 test in 34.500s

OK
test_remove_bucket_and_query (2i.cluster_ops_2i.SecondaryIndexingClusterOpsTests) ... Cluster instance shutdown with force
summary so far suite 2i.indexscans_2i.SecondaryIndexingScanTests , pass 9 , fail 0
summary so far suite 2i.indexcreatedrop_2i.SecondaryIndexingCreateDropTests , pass 2 , fail 0
testrunner logs, diags and results are available under /opt/build/testrunner/logs/testrunner-16-Apr-06_08-41-08/test_11
Logs will be stored at /opt/build/testrunner/logs/testrunner-16-Apr-06_08-41-08/test_12

./testrunner -i b/resources/dev-4-nodes-xdcr_n1ql_2i.ini makefile=True -t 2i.cluster_ops_2i.SecondaryIndexingClusterOpsTests.test_remove_bucket_and_query,groups=simple:and:no_orderby_groupby:range,dataset=default,doc-per-day=1,use_gsi_for_primary=True,use_gsi_for_secondary=True

Test Input params:
{'doc-per-day': '1', 'use_gsi_for_primary': 'True', 'use_gsi_for_secondary': 'True', 'makefile': 'True', 'conf_file': 'conf/simple_2i_n1ql.conf', 'dataset': 'default', 'cluster_name': 'dev-4-nodes-xdcr_n1ql_2i', 'ini': 'b/resources/dev-4-nodes-xdcr_n1ql_2i.ini', 'groups': 'simple:and:no_orderby_groupby:range', 'case_number': 12, 'num_nodes': 4, 'logs_folder': '/opt/build/testrunner/logs/testrunner-16-Apr-06_08-41-08/test_12', 'spec': 'simple_2i_n1ql'}
[2016-04-06 08:48:15,827] - [rest_client:1636] INFO - Node versions in cluster [u'4.5.35007-1459908767-rel-enterprise']
[2016-04-06 08:48:15,827] - [basetestcase:146] INFO - ==============  basetestcase setup was started for test #12 test_remove_bucket_and_query==============
[2016-04-06 08:48:15,839] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:48:15,843] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:48:15,848] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:48:15,853] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:48:15,863] - [basetestcase:365] INFO - sleep for 10 secs.  ...
[2016-04-06 08:48:25,867] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9000
[2016-04-06 08:48:25,870] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9000 is running
[2016-04-06 08:48:25,872] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9001
[2016-04-06 08:48:25,875] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9001 is running
[2016-04-06 08:48:25,878] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9002
[2016-04-06 08:48:25,880] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9002 is running
[2016-04-06 08:48:25,883] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9003
[2016-04-06 08:48:25,886] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9003 is running
[2016-04-06 08:48:25,886] - [basetestcase:167] INFO - initializing cluster
[2016-04-06 08:48:26,838] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:48:26,841] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:48:26,846] - [rest_client:789] INFO - /node/controller/setupServices params on 127.0.0.1: 9000:services=kv%2Cindex%2Cn1ql&password=asdasd&hostname=127.0.0.1%3A9000&user=Administrator
[2016-04-06 08:48:26,864] - [rest_client:757] ERROR - http://127.0.0.1:9000//node/controller/setupServices error 400 reason: unknown ["cannot change node services after cluster is provisioned"]
[2016-04-06 08:48:26,864] - [rest_client:794] INFO - This node is already provisioned with services, we do not consider this as failure for test case
[2016-04-06 08:48:26,864] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9000:username=Administrator&password=asdasd&port=9000
[2016-04-06 08:48:26,881] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:48:26,886] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:48:26,890] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9001:username=Administrator&password=asdasd&port=9001
[2016-04-06 08:48:26,905] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:48:26,909] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:48:26,912] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9002:username=Administrator&password=asdasd&port=9002
[2016-04-06 08:48:26,927] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:48:26,930] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:48:26,935] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9003:username=Administrator&password=asdasd&port=9003
[2016-04-06 08:48:26,948] - [basetestcase:187] INFO - done initializing cluster
[2016-04-06 08:48:27,960] - [rest_client:1772] INFO - http://127.0.0.1:9000/pools/default/buckets with param: bucketType=membase&evictionPolicy=valueOnly&threadsNumber=3&ramQuotaMB=8041&proxyPort=11211&authType=sasl&name=default&flushEnabled=1&replicaNumber=1&replicaIndex=1&saslPassword=None
[2016-04-06 08:48:27,965] - [rest_client:1794] INFO - 0.00 seconds to create bucket default
[2016-04-06 08:48:27,965] - [bucket_helper:305] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2016-04-06 08:48:29,034] - [data_helper:295] INFO - creating direct client 127.0.0.1:12000 default
[2016-04-06 08:48:29,072] - [data_helper:295] INFO - creating direct client 127.0.0.1:12000 default
[2016-04-06 08:48:29,096] - [task:288] INFO - bucket 'default' was created with per node RAM quota: 8041
[2016-04-06 08:48:29,125] - [basetestcase:257] INFO - ==============  basetestcase setup was finished for test #12 test_remove_bucket_and_query ==============
[2016-04-06 08:48:29,129] - [basetestcase:365] INFO - sleep for 10 secs.  ...
[2016-04-06 08:48:39,143] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:48:39,144] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:48:39,149] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:48:39,150] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:48:39,155] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:48:39,155] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:48:39,161] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:48:39,161] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:48:39,167] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:48:39,167] - [remote_util:168] INFO - connecting to 127.0.0.1 with username : Administrator password : asdasd ssh_key: 
[2016-04-06 08:48:39,167] - [remote_util:201] INFO - Connected to 127.0.0.1
[2016-04-06 08:48:40,079] - [basetestcase:2084] INFO - create 2016 to default documents...
[2016-04-06 08:48:40,100] - [data_helper:295] INFO - creating direct client 127.0.0.1:12000 default
[2016-04-06 08:48:42,292] - [basetestcase:2095] INFO - LOAD IS FINISHED
[2016-04-06 08:48:42,303] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:48:42,303] - [newtuq:92] INFO - ip:127.0.0.1 port:9000 ssh_username:Administrator
number of buckets 1
[2016-04-06 08:48:42,303] - [tuq_helper:419] INFO - CREATE PRIMARY INDEX ON default  USING GSI
[2016-04-06 08:48:42,304] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2016-04-06 08:48:42,306] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2016-04-06 08:48:42,341] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 32.886616ms
[2016-04-06 08:48:42,341] - [tuq_helper:75] INFO - RUN QUERY CREATE PRIMARY INDEX ON default  USING GSI
[2016-04-06 08:48:42,344] - [rest_client:2508] INFO - query params : statement=CREATE+PRIMARY+INDEX+ON+default++USING+GSI
[2016-04-06 08:48:42,856] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 509.968742ms
[2016-04-06 08:48:42,856] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2016-04-06 08:48:42,859] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2016-04-06 08:48:42,875] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 13.941723ms
[2016-04-06 08:48:42,876] - [base_2i:48] INFO - {'in_between': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}, 'initial': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}, 'after': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}, 'before': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}}
[2016-04-06 08:48:42,886] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:48:42,891] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:48:42,894] - [tuq_helper:75] INFO - RUN QUERY CREATE INDEX employeed7b5bae3d5864451bdc9b491b38f7d5djoin_yr ON default(join_yr) USING GSI  WITH {'defer_build': True}
[2016-04-06 08:48:42,896] - [rest_client:2508] INFO - query params : statement=CREATE+INDEX+employeed7b5bae3d5864451bdc9b491b38f7d5djoin_yr+ON+default%28join_yr%29+USING+GSI++WITH+%7B%27defer_build%27%3A+True%7D
[2016-04-06 08:48:42,932] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 34.402097ms
[2016-04-06 08:48:42,932] - [base_2i:102] INFO - BUILD INDEX on default(employeed7b5bae3d5864451bdc9b491b38f7d5djoin_yr) USING GSI
[2016-04-06 08:48:43,117] - [tuq_helper:75] INFO - RUN QUERY BUILD INDEX on default(employeed7b5bae3d5864451bdc9b491b38f7d5djoin_yr) USING GSI
[2016-04-06 08:48:43,120] - [rest_client:2508] INFO - query params : statement=BUILD+INDEX+on+default%28employeed7b5bae3d5864451bdc9b491b38f7d5djoin_yr%29+USING+GSI
[2016-04-06 08:48:43,132] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 10.452791ms
[2016-04-06 08:48:43,133] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employeed7b5bae3d5864451bdc9b491b38f7d5djoin_yr'
[2016-04-06 08:48:43,140] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employeed7b5bae3d5864451bdc9b491b38f7d5djoin_yr%27
[2016-04-06 08:48:43,165] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 23.20698ms
[2016-04-06 08:48:44,167] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employeed7b5bae3d5864451bdc9b491b38f7d5djoin_yr'
[2016-04-06 08:48:44,170] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employeed7b5bae3d5864451bdc9b491b38f7d5djoin_yr%27
[2016-04-06 08:48:44,188] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 16.376219ms
[2016-04-06 08:48:45,189] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employeed7b5bae3d5864451bdc9b491b38f7d5djoin_yr'
[2016-04-06 08:48:45,192] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employeed7b5bae3d5864451bdc9b491b38f7d5djoin_yr%27
[2016-04-06 08:48:45,208] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 14.370014ms
[2016-04-06 08:48:45,209] - [tuq_helper:75] INFO - RUN QUERY EXPLAIN SELECT * FROM default WHERE  join_yr > 2010 and join_yr < 2014 ORDER BY _id 
[2016-04-06 08:48:45,212] - [rest_client:2508] INFO - query params : statement=EXPLAIN+SELECT+%2A+FROM+default+WHERE++join_yr+%3E+2010+and+join_yr+%3C+2014+ORDER+BY+_id+
[2016-04-06 08:48:45,216] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 3.326291ms
[2016-04-06 08:48:45,217] - [base_2i:241] INFO - {u'status': u'success', u'metrics': {u'elapsedTime': u'3.326291ms', u'executionTime': u'3.276522ms', u'resultSize': 3247, u'resultCount': 1}, u'results': [{u'text': u'SELECT * FROM default WHERE  join_yr > 2010 and join_yr < 2014 ORDER BY _id', u'plan': {u'#operator': u'Sequence', u'~children': [{u'#operator': u'Sequence', u'~children': [{u'index': u'employeed7b5bae3d5864451bdc9b491b38f7d5djoin_yr', u'namespace': u'default', u'keyspace': u'default', u'#operator': u'IndexScan', u'spans': [{u'Range': {u'High': [u'2014'], u'Low': [u'2010'], u'Inclusion': 0}}], u'using': u'gsi', u'index_id': u'a4c79946bc76b88d'}, {u'#operator': u'Parallel', u'~child': {u'#operator': u'Sequence', u'~children': [{u'keyspace': u'default', u'#operator': u'Fetch', u'namespace': u'default'}, {u'#operator': u'Filter', u'condition': u'((2010 < (`default`.`join_yr`)) and ((`default`.`join_yr`) < 2014))'}, {u'#operator': u'InitialProject', u'result_terms': [{u'expr': u'self', u'star': True}]}]}}]}, {u'#operator': u'Order', u'sort_terms': [{u'expr': u'(`default`.`_id`)'}]}, {u'#operator': u'FinalProject'}]}}], u'requestID': u'14f65b20-d009-42cc-a638-9d169789c760', u'signature': u'json'}
[2016-04-06 08:48:45,217] - [base_2i:284] INFO - Query : SELECT * FROM default WHERE  join_yr > 2010 and join_yr < 2014 ORDER BY _id 
[2016-04-06 08:48:45,217] - [tuq_generators:66] INFO - FROM clause ===== is default
[2016-04-06 08:48:45,217] - [tuq_generators:68] INFO - WHERE clause ===== is   doc["join_yr"] > 2010 and doc["join_yr"] < 2014 
[2016-04-06 08:48:45,218] - [tuq_generators:70] INFO - UNNEST clause ===== is None
[2016-04-06 08:48:45,218] - [tuq_generators:72] INFO - SELECT clause ===== is {"*" : doc,}
[2016-04-06 08:48:45,218] - [tuq_generators:401] INFO - ORDER clause ========= is doc["_id"],
[2016-04-06 08:48:45,219] - [tuq_generators:401] INFO - ORDER clause ========= is doc["_id"],
[2016-04-06 08:48:45,268] - [tuq_generators:401] INFO - ORDER clause ========= is doc["_id"],
[2016-04-06 08:48:45,279] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM default WHERE  join_yr > 2010 and join_yr < 2014 ORDER BY _id 
[2016-04-06 08:48:45,282] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+default+WHERE++join_yr+%3E+2010+and+join_yr+%3C+2014+ORDER+BY+_id+
[2016-04-06 08:48:45,629] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 306.380798ms
[2016-04-06 08:48:45,677] - [tuq_helper:108] INFO -  Analyzing Actual Result
[2016-04-06 08:48:45,678] - [tuq_helper:110] INFO -  Analyzing Expected Result
[2016-04-06 08:48:46,239] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:48:46,239] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employeed7b5bae3d5864451bdc9b491b38f7d5djoin_yr'
[2016-04-06 08:48:46,245] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employeed7b5bae3d5864451bdc9b491b38f7d5djoin_yr%27
[2016-04-06 08:48:46,254] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 8.166975ms
[2016-04-06 08:48:46,260] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:48:46,260] - [remote_util:168] INFO - connecting to 127.0.0.1 with username : Administrator password : asdasd ssh_key: 
[2016-04-06 08:48:46,260] - [remote_util:201] INFO - Connected to 127.0.0.1
[2016-04-06 08:48:46,264] - [rest_client:1222] INFO - /diag/eval status on 127.0.0.1:9000: True content: "/opt/build/ns_server/logs/n_0" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))).
[2016-04-06 08:48:46,443] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: zgrep "panic" "/opt/build/ns_server/logs/n_0"/indexer.log* | wc -l
[2016-04-06 08:48:46,473] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:48:46,479] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:48:46,479] - [tuq_helper:392] INFO - CHECK FOR PRIMARY INDEXES
[2016-04-06 08:48:46,480] - [tuq_helper:399] INFO - DROP PRIMARY INDEX ON default USING GSI
[2016-04-06 08:48:46,480] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2016-04-06 08:48:46,482] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2016-04-06 08:48:46,490] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 5.797835ms
[2016-04-06 08:48:46,669] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: rm -f /opt/couchbase/var/lib/couchbase/meta;rm -f /tmp/log_upr_client.sock
[2016-04-06 08:48:46,696] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:48:46,697] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: killall cbq-engine
[2016-04-06 08:48:46,728] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:48:46,729] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: killall tuqtng
[2016-04-06 08:48:46,777] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:48:46,778] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: killall indexer
[2016-04-06 08:48:46,825] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:48:46,826] - [basetestcase:297] INFO - ==============  basetestcase cleanup was started for test #12 test_remove_bucket_and_query ==============
[2016-04-06 08:48:46,844] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:48:46,850] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:48:46,857] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:48:46,865] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:48:46,887] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9000
[2016-04-06 08:48:46,906] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9000 is running
[2016-04-06 08:48:46,913] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9001
[2016-04-06 08:48:46,916] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9001 is running
[2016-04-06 08:48:46,920] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9002
[2016-04-06 08:48:46,924] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9002 is running
[2016-04-06 08:48:46,932] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9003
[2016-04-06 08:48:46,936] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9003 is running
[2016-04-06 08:48:46,937] - [basetestcase:315] INFO - ==============  basetestcase cleanup was finished for test #12 test_remove_bucket_and_query ==============
ok

----------------------------------------------------------------------
Ran 1 test in 31.123s

OK
test_change_bucket_properties (2i.cluster_ops_2i.SecondaryIndexingClusterOpsTests) ... Cluster instance shutdown with force
summary so far suite 2i.indexscans_2i.SecondaryIndexingScanTests , pass 9 , fail 0
summary so far suite 2i.indexcreatedrop_2i.SecondaryIndexingCreateDropTests , pass 2 , fail 0
summary so far suite 2i.cluster_ops_2i.SecondaryIndexingClusterOpsTests , pass 1 , fail 0
testrunner logs, diags and results are available under /opt/build/testrunner/logs/testrunner-16-Apr-06_08-41-08/test_12
Logs will be stored at /opt/build/testrunner/logs/testrunner-16-Apr-06_08-41-08/test_13

./testrunner -i b/resources/dev-4-nodes-xdcr_n1ql_2i.ini makefile=True -t 2i.cluster_ops_2i.SecondaryIndexingClusterOpsTests.test_change_bucket_properties,groups=simple:and:no_orderby_groupby:range,dataset=default,doc-per-day=1,use_gsi_for_primary=True,use_gsi_for_secondary=True

Test Input params:
{'doc-per-day': '1', 'use_gsi_for_primary': 'True', 'use_gsi_for_secondary': 'True', 'makefile': 'True', 'conf_file': 'conf/simple_2i_n1ql.conf', 'dataset': 'default', 'cluster_name': 'dev-4-nodes-xdcr_n1ql_2i', 'ini': 'b/resources/dev-4-nodes-xdcr_n1ql_2i.ini', 'groups': 'simple:and:no_orderby_groupby:range', 'case_number': 13, 'num_nodes': 4, 'logs_folder': '/opt/build/testrunner/logs/testrunner-16-Apr-06_08-41-08/test_13', 'spec': 'simple_2i_n1ql'}
[2016-04-06 08:48:46,974] - [rest_client:1636] INFO - Node versions in cluster [u'4.5.35007-1459908767-rel-enterprise']
[2016-04-06 08:48:46,974] - [basetestcase:146] INFO - ==============  basetestcase setup was started for test #13 test_change_bucket_properties==============
[2016-04-06 08:48:46,994] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:48:46,999] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:48:47,003] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:48:47,008] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:48:47,019] - [basetestcase:365] INFO - sleep for 10 secs.  ...
[2016-04-06 08:48:57,028] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9000
[2016-04-06 08:48:57,031] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9000 is running
[2016-04-06 08:48:57,034] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9001
[2016-04-06 08:48:57,038] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9001 is running
[2016-04-06 08:48:57,040] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9002
[2016-04-06 08:48:57,043] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9002 is running
[2016-04-06 08:48:57,046] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9003
[2016-04-06 08:48:57,048] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9003 is running
[2016-04-06 08:48:57,049] - [basetestcase:167] INFO - initializing cluster
[2016-04-06 08:48:57,981] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:48:57,984] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:48:57,989] - [rest_client:789] INFO - /node/controller/setupServices params on 127.0.0.1: 9000:services=kv%2Cindex%2Cn1ql&password=asdasd&hostname=127.0.0.1%3A9000&user=Administrator
[2016-04-06 08:48:58,006] - [rest_client:757] ERROR - http://127.0.0.1:9000//node/controller/setupServices error 400 reason: unknown ["cannot change node services after cluster is provisioned"]
[2016-04-06 08:48:58,006] - [rest_client:794] INFO - This node is already provisioned with services, we do not consider this as failure for test case
[2016-04-06 08:48:58,006] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9000:username=Administrator&password=asdasd&port=9000
[2016-04-06 08:48:58,027] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:48:58,032] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:48:58,039] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9001:username=Administrator&password=asdasd&port=9001
[2016-04-06 08:48:58,057] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:48:58,061] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:48:58,065] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9002:username=Administrator&password=asdasd&port=9002
[2016-04-06 08:48:58,080] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:48:58,083] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:48:58,091] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9003:username=Administrator&password=asdasd&port=9003
[2016-04-06 08:48:58,100] - [basetestcase:187] INFO - done initializing cluster
[2016-04-06 08:48:59,109] - [rest_client:1772] INFO - http://127.0.0.1:9000/pools/default/buckets with param: bucketType=membase&evictionPolicy=valueOnly&threadsNumber=3&ramQuotaMB=8041&proxyPort=11211&authType=sasl&name=default&flushEnabled=1&replicaNumber=1&replicaIndex=1&saslPassword=None
[2016-04-06 08:48:59,114] - [rest_client:1794] INFO - 0.00 seconds to create bucket default
[2016-04-06 08:48:59,114] - [bucket_helper:305] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2016-04-06 08:49:00,185] - [data_helper:295] INFO - creating direct client 127.0.0.1:12000 default
[2016-04-06 08:49:00,217] - [data_helper:295] INFO - creating direct client 127.0.0.1:12000 default
[2016-04-06 08:49:00,241] - [task:288] INFO - bucket 'default' was created with per node RAM quota: 8041
[2016-04-06 08:49:00,274] - [basetestcase:257] INFO - ==============  basetestcase setup was finished for test #13 test_change_bucket_properties ==============
[2016-04-06 08:49:00,280] - [basetestcase:365] INFO - sleep for 10 secs.  ...
[2016-04-06 08:49:10,293] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:49:10,293] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:49:10,299] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:49:10,299] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:49:10,305] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:49:10,305] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:49:10,310] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:49:10,310] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:49:10,317] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:49:10,317] - [remote_util:168] INFO - connecting to 127.0.0.1 with username : Administrator password : asdasd ssh_key: 
[2016-04-06 08:49:10,317] - [remote_util:201] INFO - Connected to 127.0.0.1
[2016-04-06 08:49:11,267] - [basetestcase:2084] INFO - create 2016 to default documents...
[2016-04-06 08:49:11,285] - [data_helper:295] INFO - creating direct client 127.0.0.1:12000 default
[2016-04-06 08:49:13,435] - [basetestcase:2095] INFO - LOAD IS FINISHED
[2016-04-06 08:49:13,445] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:49:13,446] - [newtuq:92] INFO - ip:127.0.0.1 port:9000 ssh_username:Administrator
number of buckets 1
[2016-04-06 08:49:13,446] - [tuq_helper:419] INFO - CREATE PRIMARY INDEX ON default  USING GSI
[2016-04-06 08:49:13,446] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2016-04-06 08:49:13,450] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2016-04-06 08:49:13,495] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 43.522391ms
[2016-04-06 08:49:13,496] - [tuq_helper:75] INFO - RUN QUERY CREATE PRIMARY INDEX ON default  USING GSI
[2016-04-06 08:49:13,499] - [rest_client:2508] INFO - query params : statement=CREATE+PRIMARY+INDEX+ON+default++USING+GSI
[2016-04-06 08:49:14,000] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 499.306071ms
[2016-04-06 08:49:14,000] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2016-04-06 08:49:14,003] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2016-04-06 08:49:14,020] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 15.148907ms
[2016-04-06 08:49:14,021] - [base_2i:48] INFO - {'in_between': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}, 'initial': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}, 'after': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}, 'before': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}}
[2016-04-06 08:49:14,031] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:49:14,036] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:49:14,038] - [tuq_helper:75] INFO - RUN QUERY CREATE INDEX employee4740847238cb495aa2fd4bbd652ccb6fjoin_yr ON default(join_yr) USING GSI  WITH {'defer_build': True}
[2016-04-06 08:49:14,041] - [rest_client:2508] INFO - query params : statement=CREATE+INDEX+employee4740847238cb495aa2fd4bbd652ccb6fjoin_yr+ON+default%28join_yr%29+USING+GSI++WITH+%7B%27defer_build%27%3A+True%7D
[2016-04-06 08:49:14,078] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 35.239324ms
[2016-04-06 08:49:14,078] - [base_2i:102] INFO - BUILD INDEX on default(employee4740847238cb495aa2fd4bbd652ccb6fjoin_yr) USING GSI
[2016-04-06 08:49:14,262] - [tuq_helper:75] INFO - RUN QUERY BUILD INDEX on default(employee4740847238cb495aa2fd4bbd652ccb6fjoin_yr) USING GSI
[2016-04-06 08:49:14,265] - [rest_client:2508] INFO - query params : statement=BUILD+INDEX+on+default%28employee4740847238cb495aa2fd4bbd652ccb6fjoin_yr%29+USING+GSI
[2016-04-06 08:49:14,282] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 15.178422ms
[2016-04-06 08:49:14,283] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee4740847238cb495aa2fd4bbd652ccb6fjoin_yr'
[2016-04-06 08:49:14,286] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee4740847238cb495aa2fd4bbd652ccb6fjoin_yr%27
[2016-04-06 08:49:14,310] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 22.15341ms
[2016-04-06 08:49:15,311] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee4740847238cb495aa2fd4bbd652ccb6fjoin_yr'
[2016-04-06 08:49:15,315] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee4740847238cb495aa2fd4bbd652ccb6fjoin_yr%27
[2016-04-06 08:49:15,330] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 13.407284ms
[2016-04-06 08:49:16,330] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee4740847238cb495aa2fd4bbd652ccb6fjoin_yr'
[2016-04-06 08:49:16,334] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee4740847238cb495aa2fd4bbd652ccb6fjoin_yr%27
[2016-04-06 08:49:16,352] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 16.762516ms
[2016-04-06 08:49:16,352] - [tuq_helper:75] INFO - RUN QUERY EXPLAIN SELECT * FROM default WHERE  join_yr > 2010 and join_yr < 2014 ORDER BY _id 
[2016-04-06 08:49:16,355] - [rest_client:2508] INFO - query params : statement=EXPLAIN+SELECT+%2A+FROM+default+WHERE++join_yr+%3E+2010+and+join_yr+%3C+2014+ORDER+BY+_id+
[2016-04-06 08:49:16,359] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 3.333524ms
[2016-04-06 08:49:16,360] - [base_2i:241] INFO - {u'status': u'success', u'metrics': {u'elapsedTime': u'3.333524ms', u'executionTime': u'3.283801ms', u'resultSize': 3247, u'resultCount': 1}, u'results': [{u'text': u'SELECT * FROM default WHERE  join_yr > 2010 and join_yr < 2014 ORDER BY _id', u'plan': {u'#operator': u'Sequence', u'~children': [{u'#operator': u'Sequence', u'~children': [{u'index': u'employee4740847238cb495aa2fd4bbd652ccb6fjoin_yr', u'namespace': u'default', u'keyspace': u'default', u'#operator': u'IndexScan', u'spans': [{u'Range': {u'High': [u'2014'], u'Low': [u'2010'], u'Inclusion': 0}}], u'using': u'gsi', u'index_id': u'35adea8d6a08f57d'}, {u'#operator': u'Parallel', u'~child': {u'#operator': u'Sequence', u'~children': [{u'keyspace': u'default', u'#operator': u'Fetch', u'namespace': u'default'}, {u'#operator': u'Filter', u'condition': u'((2010 < (`default`.`join_yr`)) and ((`default`.`join_yr`) < 2014))'}, {u'#operator': u'InitialProject', u'result_terms': [{u'expr': u'self', u'star': True}]}]}}]}, {u'#operator': u'Order', u'sort_terms': [{u'expr': u'(`default`.`_id`)'}]}, {u'#operator': u'FinalProject'}]}}], u'requestID': u'6e80e8c2-0454-4b43-87e6-3844f50b032e', u'signature': u'json'}
[2016-04-06 08:49:16,360] - [base_2i:284] INFO - Query : SELECT * FROM default WHERE  join_yr > 2010 and join_yr < 2014 ORDER BY _id 
[2016-04-06 08:49:16,360] - [tuq_generators:66] INFO - FROM clause ===== is default
[2016-04-06 08:49:16,360] - [tuq_generators:68] INFO - WHERE clause ===== is   doc["join_yr"] > 2010 and doc["join_yr"] < 2014 
[2016-04-06 08:49:16,361] - [tuq_generators:70] INFO - UNNEST clause ===== is None
[2016-04-06 08:49:16,361] - [tuq_generators:72] INFO - SELECT clause ===== is {"*" : doc,}
[2016-04-06 08:49:16,361] - [tuq_generators:401] INFO - ORDER clause ========= is doc["_id"],
[2016-04-06 08:49:16,361] - [tuq_generators:401] INFO - ORDER clause ========= is doc["_id"],
[2016-04-06 08:49:16,409] - [tuq_generators:401] INFO - ORDER clause ========= is doc["_id"],
[2016-04-06 08:49:16,422] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM default WHERE  join_yr > 2010 and join_yr < 2014 ORDER BY _id 
[2016-04-06 08:49:16,425] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+default+WHERE++join_yr+%3E+2010+and+join_yr+%3C+2014+ORDER+BY+_id+
[2016-04-06 08:49:16,758] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 310.617774ms
[2016-04-06 08:49:16,809] - [tuq_helper:108] INFO -  Analyzing Actual Result
[2016-04-06 08:49:16,810] - [tuq_helper:110] INFO -  Analyzing Expected Result
[2016-04-06 08:49:16,819] - [rest_client:1828] INFO - http://127.0.0.1:9000/pools/default/buckets/default with param: 
[2016-04-06 08:49:16,825] - [rest_client:1832] INFO - bucket default updated
[2016-04-06 08:49:16,826] - [tuq_helper:75] INFO - RUN QUERY EXPLAIN SELECT * FROM default WHERE  join_yr > 2010 and join_yr < 2014 ORDER BY _id 
[2016-04-06 08:49:16,828] - [rest_client:2508] INFO - query params : statement=EXPLAIN+SELECT+%2A+FROM+default+WHERE++join_yr+%3E+2010+and+join_yr+%3C+2014+ORDER+BY+_id+
[2016-04-06 08:49:16,834] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 3.876095ms
[2016-04-06 08:49:16,834] - [base_2i:241] INFO - {u'status': u'success', u'metrics': {u'elapsedTime': u'3.876095ms', u'executionTime': u'3.793897ms', u'resultSize': 3247, u'resultCount': 1}, u'results': [{u'text': u'SELECT * FROM default WHERE  join_yr > 2010 and join_yr < 2014 ORDER BY _id', u'plan': {u'#operator': u'Sequence', u'~children': [{u'#operator': u'Sequence', u'~children': [{u'index': u'employee4740847238cb495aa2fd4bbd652ccb6fjoin_yr', u'namespace': u'default', u'keyspace': u'default', u'#operator': u'IndexScan', u'spans': [{u'Range': {u'High': [u'2014'], u'Low': [u'2010'], u'Inclusion': 0}}], u'using': u'gsi', u'index_id': u'35adea8d6a08f57d'}, {u'#operator': u'Parallel', u'~child': {u'#operator': u'Sequence', u'~children': [{u'keyspace': u'default', u'#operator': u'Fetch', u'namespace': u'default'}, {u'#operator': u'Filter', u'condition': u'((2010 < (`default`.`join_yr`)) and ((`default`.`join_yr`) < 2014))'}, {u'#operator': u'InitialProject', u'result_terms': [{u'expr': u'self', u'star': True}]}]}}]}, {u'#operator': u'Order', u'sort_terms': [{u'expr': u'(`default`.`_id`)'}]}, {u'#operator': u'FinalProject'}]}}], u'requestID': u'd357653e-c76f-4284-a824-9967f7b74355', u'signature': u'json'}
[2016-04-06 08:49:16,835] - [base_2i:284] INFO - Query : SELECT * FROM default WHERE  join_yr > 2010 and join_yr < 2014 ORDER BY _id 
[2016-04-06 08:49:16,835] - [tuq_generators:66] INFO - FROM clause ===== is default
[2016-04-06 08:49:16,835] - [tuq_generators:68] INFO - WHERE clause ===== is   doc["join_yr"] > 2010 and doc["join_yr"] < 2014 
[2016-04-06 08:49:16,835] - [tuq_generators:70] INFO - UNNEST clause ===== is None
[2016-04-06 08:49:16,835] - [tuq_generators:72] INFO - SELECT clause ===== is {"*" : doc,}
[2016-04-06 08:49:16,836] - [tuq_generators:401] INFO - ORDER clause ========= is doc["_id"],
[2016-04-06 08:49:16,836] - [tuq_generators:401] INFO - ORDER clause ========= is doc["_id"],
[2016-04-06 08:49:16,886] - [tuq_generators:401] INFO - ORDER clause ========= is doc["_id"],
[2016-04-06 08:49:16,897] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM default WHERE  join_yr > 2010 and join_yr < 2014 ORDER BY _id 
[2016-04-06 08:49:16,900] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+default+WHERE++join_yr+%3E+2010+and+join_yr+%3C+2014+ORDER+BY+_id+
[2016-04-06 08:49:17,232] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 310.062468ms
[2016-04-06 08:49:17,279] - [tuq_helper:108] INFO -  Analyzing Actual Result
[2016-04-06 08:49:17,280] - [tuq_helper:110] INFO -  Analyzing Expected Result
[2016-04-06 08:49:17,285] - [tuq_helper:75] INFO - RUN QUERY DROP INDEX default.employee4740847238cb495aa2fd4bbd652ccb6fjoin_yr USING GSI
[2016-04-06 08:49:17,288] - [rest_client:2508] INFO - query params : statement=DROP+INDEX+default.employee4740847238cb495aa2fd4bbd652ccb6fjoin_yr+USING+GSI
[2016-04-06 08:49:17,316] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 23.250629ms
[2016-04-06 08:49:17,316] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee4740847238cb495aa2fd4bbd652ccb6fjoin_yr'
[2016-04-06 08:49:17,320] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee4740847238cb495aa2fd4bbd652ccb6fjoin_yr%27
[2016-04-06 08:49:17,334] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 12.53395ms
[2016-04-06 08:49:17,339] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:49:17,339] - [remote_util:168] INFO - connecting to 127.0.0.1 with username : Administrator password : asdasd ssh_key: 
[2016-04-06 08:49:17,339] - [remote_util:201] INFO - Connected to 127.0.0.1
[2016-04-06 08:49:17,346] - [rest_client:1222] INFO - /diag/eval status on 127.0.0.1:9000: True content: "/opt/build/ns_server/logs/n_0" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))).
[2016-04-06 08:49:17,538] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: zgrep "panic" "/opt/build/ns_server/logs/n_0"/indexer.log* | wc -l
[2016-04-06 08:49:17,579] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:49:17,588] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:49:17,588] - [tuq_helper:392] INFO - CHECK FOR PRIMARY INDEXES
[2016-04-06 08:49:17,588] - [tuq_helper:399] INFO - DROP PRIMARY INDEX ON default USING GSI
[2016-04-06 08:49:17,588] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2016-04-06 08:49:17,591] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2016-04-06 08:49:17,612] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 18.905342ms
[2016-04-06 08:49:17,612] - [tuq_helper:75] INFO - RUN QUERY DROP PRIMARY INDEX ON default USING GSI
[2016-04-06 08:49:17,614] - [rest_client:2508] INFO - query params : statement=DROP+PRIMARY+INDEX+ON+default+USING+GSI
[2016-04-06 08:49:17,641] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 25.174108ms
[2016-04-06 08:49:17,817] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: rm -f /opt/couchbase/var/lib/couchbase/meta;rm -f /tmp/log_upr_client.sock
[2016-04-06 08:49:17,843] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:49:17,844] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: killall cbq-engine
[2016-04-06 08:49:17,874] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:49:17,874] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: killall tuqtng
[2016-04-06 08:49:17,914] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:49:17,915] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: killall indexer
[2016-04-06 08:49:17,955] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:49:17,956] - [basetestcase:297] INFO - ==============  basetestcase cleanup was started for test #13 test_change_bucket_properties ==============
[2016-04-06 08:49:17,980] - [bucket_helper:137] INFO - deleting existing buckets [u'default'] on 127.0.0.1
[2016-04-06 08:49:17,981] - [bucket_helper:139] INFO - remove bucket default ...
[2016-04-06 08:49:19,425] - [bucket_helper:153] INFO - deleted bucket : default from 127.0.0.1
[2016-04-06 08:49:19,425] - [bucket_helper:226] INFO - waiting for bucket deletion to complete....
[2016-04-06 08:49:19,428] - [rest_client:127] INFO - node 127.0.0.1 existing buckets : []
[2016-04-06 08:49:19,434] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:49:19,439] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:49:19,445] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:49:19,460] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9000
[2016-04-06 08:49:19,463] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9000 is running
[2016-04-06 08:49:19,466] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9001
[2016-04-06 08:49:19,469] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9001 is running
[2016-04-06 08:49:19,472] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9002
[2016-04-06 08:49:19,475] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9002 is running
[2016-04-06 08:49:19,477] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9003
[2016-04-06 08:49:19,480] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9003 is running
[2016-04-06 08:49:19,480] - [basetestcase:315] INFO - ==============  basetestcase cleanup was finished for test #13 test_change_bucket_properties ==============
ok

----------------------------------------------------------------------
Ran 1 test in 32.521s

OK
test_flush_bucket_and_query (2i.cluster_ops_2i.SecondaryIndexingClusterOpsTests) ... Cluster instance shutdown with force
summary so far suite 2i.indexscans_2i.SecondaryIndexingScanTests , pass 9 , fail 0
summary so far suite 2i.indexcreatedrop_2i.SecondaryIndexingCreateDropTests , pass 2 , fail 0
summary so far suite 2i.cluster_ops_2i.SecondaryIndexingClusterOpsTests , pass 2 , fail 0
testrunner logs, diags and results are available under /opt/build/testrunner/logs/testrunner-16-Apr-06_08-41-08/test_13
Logs will be stored at /opt/build/testrunner/logs/testrunner-16-Apr-06_08-41-08/test_14

./testrunner -i b/resources/dev-4-nodes-xdcr_n1ql_2i.ini makefile=True -t 2i.cluster_ops_2i.SecondaryIndexingClusterOpsTests.test_flush_bucket_and_query,groups=simple:and:no_orderby_groupby:range,dataset=default,doc-per-day=1,use_gsi_for_primary=True,use_gsi_for_secondary=True

Test Input params:
{'doc-per-day': '1', 'use_gsi_for_primary': 'True', 'use_gsi_for_secondary': 'True', 'makefile': 'True', 'conf_file': 'conf/simple_2i_n1ql.conf', 'dataset': 'default', 'cluster_name': 'dev-4-nodes-xdcr_n1ql_2i', 'ini': 'b/resources/dev-4-nodes-xdcr_n1ql_2i.ini', 'groups': 'simple:and:no_orderby_groupby:range', 'case_number': 14, 'num_nodes': 4, 'logs_folder': '/opt/build/testrunner/logs/testrunner-16-Apr-06_08-41-08/test_14', 'spec': 'simple_2i_n1ql'}
[2016-04-06 08:49:19,505] - [rest_client:1636] INFO - Node versions in cluster [u'4.5.35007-1459908767-rel-enterprise']
[2016-04-06 08:49:19,505] - [basetestcase:146] INFO - ==============  basetestcase setup was started for test #14 test_flush_bucket_and_query==============
[2016-04-06 08:49:19,516] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:49:19,520] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:49:19,525] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:49:19,529] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:49:19,543] - [basetestcase:365] INFO - sleep for 10 secs.  ...
[2016-04-06 08:49:29,547] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9000
[2016-04-06 08:49:29,549] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9000 is running
[2016-04-06 08:49:29,552] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9001
[2016-04-06 08:49:29,555] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9001 is running
[2016-04-06 08:49:29,558] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9002
[2016-04-06 08:49:29,560] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9002 is running
[2016-04-06 08:49:29,563] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9003
[2016-04-06 08:49:29,565] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9003 is running
[2016-04-06 08:49:29,565] - [basetestcase:167] INFO - initializing cluster
[2016-04-06 08:49:30,521] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:49:30,524] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:49:30,532] - [rest_client:789] INFO - /node/controller/setupServices params on 127.0.0.1: 9000:services=kv%2Cindex%2Cn1ql&password=asdasd&hostname=127.0.0.1%3A9000&user=Administrator
[2016-04-06 08:49:30,535] - [rest_client:757] ERROR - http://127.0.0.1:9000//node/controller/setupServices error 400 reason: unknown ["cannot change node services after cluster is provisioned"]
[2016-04-06 08:49:30,535] - [rest_client:794] INFO - This node is already provisioned with services, we do not consider this as failure for test case
[2016-04-06 08:49:30,536] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9000:username=Administrator&password=asdasd&port=9000
[2016-04-06 08:49:30,573] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:49:30,577] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:49:30,581] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9001:username=Administrator&password=asdasd&port=9001
[2016-04-06 08:49:30,597] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:49:30,601] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:49:30,605] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9002:username=Administrator&password=asdasd&port=9002
[2016-04-06 08:49:30,621] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:49:30,624] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:49:30,629] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9003:username=Administrator&password=asdasd&port=9003
[2016-04-06 08:49:30,638] - [basetestcase:187] INFO - done initializing cluster
[2016-04-06 08:49:31,648] - [rest_client:1772] INFO - http://127.0.0.1:9000/pools/default/buckets with param: bucketType=membase&evictionPolicy=valueOnly&threadsNumber=3&ramQuotaMB=8041&proxyPort=11211&authType=sasl&name=default&flushEnabled=1&replicaNumber=1&replicaIndex=1&saslPassword=None
[2016-04-06 08:49:31,652] - [rest_client:1794] INFO - 0.00 seconds to create bucket default
[2016-04-06 08:49:31,653] - [bucket_helper:305] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2016-04-06 08:49:32,722] - [data_helper:295] INFO - creating direct client 127.0.0.1:12000 default
[2016-04-06 08:49:32,750] - [data_helper:295] INFO - creating direct client 127.0.0.1:12000 default
[2016-04-06 08:49:32,775] - [task:288] INFO - bucket 'default' was created with per node RAM quota: 8041
[2016-04-06 08:49:32,814] - [basetestcase:257] INFO - ==============  basetestcase setup was finished for test #14 test_flush_bucket_and_query ==============
[2016-04-06 08:49:32,819] - [basetestcase:365] INFO - sleep for 10 secs.  ...
[2016-04-06 08:49:42,826] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:49:42,827] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:49:42,833] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:49:42,833] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:49:42,839] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:49:42,839] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:49:42,845] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:49:42,845] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:49:42,850] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:49:42,850] - [remote_util:168] INFO - connecting to 127.0.0.1 with username : Administrator password : asdasd ssh_key: 
[2016-04-06 08:49:42,850] - [remote_util:201] INFO - Connected to 127.0.0.1
[2016-04-06 08:49:43,782] - [basetestcase:2084] INFO - create 2016 to default documents...
[2016-04-06 08:49:43,803] - [data_helper:295] INFO - creating direct client 127.0.0.1:12000 default
[2016-04-06 08:49:45,958] - [basetestcase:2095] INFO - LOAD IS FINISHED
[2016-04-06 08:49:45,969] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:49:45,970] - [newtuq:92] INFO - ip:127.0.0.1 port:9000 ssh_username:Administrator
number of buckets 1
[2016-04-06 08:49:45,970] - [tuq_helper:419] INFO - CREATE PRIMARY INDEX ON default  USING GSI
[2016-04-06 08:49:45,970] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2016-04-06 08:49:45,973] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2016-04-06 08:49:46,011] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 36.237268ms
[2016-04-06 08:49:46,011] - [tuq_helper:75] INFO - RUN QUERY CREATE PRIMARY INDEX ON default  USING GSI
[2016-04-06 08:49:46,014] - [rest_client:2508] INFO - query params : statement=CREATE+PRIMARY+INDEX+ON+default++USING+GSI
[2016-04-06 08:49:46,511] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 494.62002ms
[2016-04-06 08:49:46,511] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2016-04-06 08:49:46,514] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2016-04-06 08:49:46,532] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 16.389599ms
[2016-04-06 08:49:46,533] - [base_2i:48] INFO - {'in_between': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}, 'initial': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}, 'after': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}, 'before': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}}
[2016-04-06 08:49:46,548] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:49:46,552] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:49:46,556] - [tuq_helper:75] INFO - RUN QUERY CREATE INDEX employeece800aa401d343ba96b5eeee76d9b1b1join_yr ON default(join_yr) USING GSI  WITH {'defer_build': True}
[2016-04-06 08:49:46,560] - [rest_client:2508] INFO - query params : statement=CREATE+INDEX+employeece800aa401d343ba96b5eeee76d9b1b1join_yr+ON+default%28join_yr%29+USING+GSI++WITH+%7B%27defer_build%27%3A+True%7D
[2016-04-06 08:49:46,600] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 37.39788ms
[2016-04-06 08:49:46,600] - [base_2i:102] INFO - BUILD INDEX on default(employeece800aa401d343ba96b5eeee76d9b1b1join_yr) USING GSI
[2016-04-06 08:49:46,795] - [tuq_helper:75] INFO - RUN QUERY BUILD INDEX on default(employeece800aa401d343ba96b5eeee76d9b1b1join_yr) USING GSI
[2016-04-06 08:49:46,798] - [rest_client:2508] INFO - query params : statement=BUILD+INDEX+on+default%28employeece800aa401d343ba96b5eeee76d9b1b1join_yr%29+USING+GSI
[2016-04-06 08:49:46,813] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 13.213781ms
[2016-04-06 08:49:46,813] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employeece800aa401d343ba96b5eeee76d9b1b1join_yr'
[2016-04-06 08:49:46,817] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employeece800aa401d343ba96b5eeee76d9b1b1join_yr%27
[2016-04-06 08:49:46,837] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 17.559114ms
[2016-04-06 08:49:47,838] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employeece800aa401d343ba96b5eeee76d9b1b1join_yr'
[2016-04-06 08:49:47,841] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employeece800aa401d343ba96b5eeee76d9b1b1join_yr%27
[2016-04-06 08:49:47,860] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 16.774083ms
[2016-04-06 08:49:48,861] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employeece800aa401d343ba96b5eeee76d9b1b1join_yr'
[2016-04-06 08:49:48,864] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employeece800aa401d343ba96b5eeee76d9b1b1join_yr%27
[2016-04-06 08:49:48,880] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 14.683002ms
[2016-04-06 08:49:48,880] - [tuq_helper:75] INFO - RUN QUERY EXPLAIN SELECT * FROM default WHERE  join_yr > 2010 and join_yr < 2014 ORDER BY _id 
[2016-04-06 08:49:48,883] - [rest_client:2508] INFO - query params : statement=EXPLAIN+SELECT+%2A+FROM+default+WHERE++join_yr+%3E+2010+and+join_yr+%3C+2014+ORDER+BY+_id+
[2016-04-06 08:49:48,887] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 2.444803ms
[2016-04-06 08:49:48,887] - [base_2i:241] INFO - {u'status': u'success', u'metrics': {u'elapsedTime': u'2.444803ms', u'executionTime': u'2.40164ms', u'resultSize': 3247, u'resultCount': 1}, u'results': [{u'text': u'SELECT * FROM default WHERE  join_yr > 2010 and join_yr < 2014 ORDER BY _id', u'plan': {u'#operator': u'Sequence', u'~children': [{u'#operator': u'Sequence', u'~children': [{u'index': u'employeece800aa401d343ba96b5eeee76d9b1b1join_yr', u'namespace': u'default', u'keyspace': u'default', u'#operator': u'IndexScan', u'spans': [{u'Range': {u'High': [u'2014'], u'Low': [u'2010'], u'Inclusion': 0}}], u'using': u'gsi', u'index_id': u'db75c17adff6a04b'}, {u'#operator': u'Parallel', u'~child': {u'#operator': u'Sequence', u'~children': [{u'keyspace': u'default', u'#operator': u'Fetch', u'namespace': u'default'}, {u'#operator': u'Filter', u'condition': u'((2010 < (`default`.`join_yr`)) and ((`default`.`join_yr`) < 2014))'}, {u'#operator': u'InitialProject', u'result_terms': [{u'expr': u'self', u'star': True}]}]}}]}, {u'#operator': u'Order', u'sort_terms': [{u'expr': u'(`default`.`_id`)'}]}, {u'#operator': u'FinalProject'}]}}], u'requestID': u'318bfe55-839a-4320-99e2-97fb36a12a25', u'signature': u'json'}
[2016-04-06 08:49:48,888] - [base_2i:284] INFO - Query : SELECT * FROM default WHERE  join_yr > 2010 and join_yr < 2014 ORDER BY _id 
[2016-04-06 08:49:48,888] - [tuq_generators:66] INFO - FROM clause ===== is default
[2016-04-06 08:49:48,888] - [tuq_generators:68] INFO - WHERE clause ===== is   doc["join_yr"] > 2010 and doc["join_yr"] < 2014 
[2016-04-06 08:49:48,888] - [tuq_generators:70] INFO - UNNEST clause ===== is None
[2016-04-06 08:49:48,888] - [tuq_generators:72] INFO - SELECT clause ===== is {"*" : doc,}
[2016-04-06 08:49:48,888] - [tuq_generators:401] INFO - ORDER clause ========= is doc["_id"],
[2016-04-06 08:49:48,889] - [tuq_generators:401] INFO - ORDER clause ========= is doc["_id"],
[2016-04-06 08:49:48,937] - [tuq_generators:401] INFO - ORDER clause ========= is doc["_id"],
[2016-04-06 08:49:48,948] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM default WHERE  join_yr > 2010 and join_yr < 2014 ORDER BY _id 
[2016-04-06 08:49:48,951] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+default+WHERE++join_yr+%3E+2010+and+join_yr+%3C+2014+ORDER+BY+_id+
[2016-04-06 08:49:49,265] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 291.152127ms
[2016-04-06 08:49:49,313] - [tuq_helper:108] INFO -  Analyzing Actual Result
[2016-04-06 08:49:49,314] - [tuq_helper:110] INFO -  Analyzing Expected Result
[2016-04-06 08:49:49,498] - [rest_client:2411] INFO - Flush for bucket 'default' was triggered
[2016-04-06 08:49:49,498] - [basetestcase:365] INFO - sleep for 2 secs.  ...
[2016-04-06 08:49:51,501] - [base_2i:319] INFO - Query : SELECT * FROM default WHERE  join_yr > 2010 and join_yr < 2014 ORDER BY _id 
[2016-04-06 08:49:51,501] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM default WHERE  join_yr > 2010 and join_yr < 2014 ORDER BY _id 
[2016-04-06 08:49:51,504] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+default+WHERE++join_yr+%3E+2010+and+join_yr+%3C+2014+ORDER+BY+_id+
[2016-04-06 08:49:51,509] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 3.641337ms
[2016-04-06 08:49:51,515] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:49:51,515] - [remote_util:168] INFO - connecting to 127.0.0.1 with username : Administrator password : asdasd ssh_key: 
[2016-04-06 08:49:51,515] - [remote_util:201] INFO - Connected to 127.0.0.1
[2016-04-06 08:49:51,520] - [rest_client:1222] INFO - /diag/eval status on 127.0.0.1:9000: True content: "/opt/build/ns_server/logs/n_0" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))).
[2016-04-06 08:49:51,703] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: zgrep "panic" "/opt/build/ns_server/logs/n_0"/indexer.log* | wc -l
[2016-04-06 08:49:51,731] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:49:51,738] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:49:51,738] - [tuq_helper:392] INFO - CHECK FOR PRIMARY INDEXES
[2016-04-06 08:49:51,738] - [tuq_helper:399] INFO - DROP PRIMARY INDEX ON default USING GSI
[2016-04-06 08:49:51,738] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2016-04-06 08:49:51,741] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2016-04-06 08:49:51,755] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 12.036824ms
[2016-04-06 08:49:51,755] - [tuq_helper:75] INFO - RUN QUERY DROP PRIMARY INDEX ON default USING GSI
[2016-04-06 08:49:51,758] - [rest_client:2508] INFO - query params : statement=DROP+PRIMARY+INDEX+ON+default+USING+GSI
[2016-04-06 08:49:51,786] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 26.512758ms
[2016-04-06 08:49:51,967] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: rm -f /opt/couchbase/var/lib/couchbase/meta;rm -f /tmp/log_upr_client.sock
[2016-04-06 08:49:51,993] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:49:51,993] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: killall cbq-engine
[2016-04-06 08:49:52,022] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:49:52,023] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: killall tuqtng
[2016-04-06 08:49:52,063] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:49:52,063] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: killall indexer
[2016-04-06 08:49:52,095] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:49:52,095] - [basetestcase:297] INFO - ==============  basetestcase cleanup was started for test #14 test_flush_bucket_and_query ==============
[2016-04-06 08:49:52,118] - [bucket_helper:137] INFO - deleting existing buckets [u'default'] on 127.0.0.1
[2016-04-06 08:49:52,118] - [bucket_helper:139] INFO - remove bucket default ...
[2016-04-06 08:49:53,912] - [bucket_helper:153] INFO - deleted bucket : default from 127.0.0.1
[2016-04-06 08:49:53,912] - [bucket_helper:226] INFO - waiting for bucket deletion to complete....
[2016-04-06 08:49:53,914] - [rest_client:127] INFO - node 127.0.0.1 existing buckets : []
[2016-04-06 08:49:53,920] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:49:53,926] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:49:53,932] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:49:53,952] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9000
[2016-04-06 08:49:53,955] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9000 is running
[2016-04-06 08:49:53,958] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9001
[2016-04-06 08:49:53,961] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9001 is running
[2016-04-06 08:49:53,963] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9002
[2016-04-06 08:49:53,965] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9002 is running
[2016-04-06 08:49:53,968] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9003
[2016-04-06 08:49:53,971] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9003 is running
[2016-04-06 08:49:53,971] - [basetestcase:315] INFO - ==============  basetestcase cleanup was finished for test #14 test_flush_bucket_and_query ==============
ok

----------------------------------------------------------------------
Ran 1 test in 34.477s

OK
test_delete_create_bucket_and_query (2i.cluster_ops_2i.SecondaryIndexingClusterOpsTests) ... Cluster instance shutdown with force
summary so far suite 2i.indexscans_2i.SecondaryIndexingScanTests , pass 9 , fail 0
summary so far suite 2i.indexcreatedrop_2i.SecondaryIndexingCreateDropTests , pass 2 , fail 0
summary so far suite 2i.cluster_ops_2i.SecondaryIndexingClusterOpsTests , pass 3 , fail 0
testrunner logs, diags and results are available under /opt/build/testrunner/logs/testrunner-16-Apr-06_08-41-08/test_14
Logs will be stored at /opt/build/testrunner/logs/testrunner-16-Apr-06_08-41-08/test_15

./testrunner -i b/resources/dev-4-nodes-xdcr_n1ql_2i.ini makefile=True -t 2i.cluster_ops_2i.SecondaryIndexingClusterOpsTests.test_delete_create_bucket_and_query,groups=simple:and:no_orderby_groupby:range,dataset=default,doc-per-day=1,use_gsi_for_primary=True,use_gsi_for_secondary=True

Test Input params:
{'doc-per-day': '1', 'use_gsi_for_primary': 'True', 'use_gsi_for_secondary': 'True', 'makefile': 'True', 'conf_file': 'conf/simple_2i_n1ql.conf', 'dataset': 'default', 'cluster_name': 'dev-4-nodes-xdcr_n1ql_2i', 'ini': 'b/resources/dev-4-nodes-xdcr_n1ql_2i.ini', 'groups': 'simple:and:no_orderby_groupby:range', 'case_number': 15, 'num_nodes': 4, 'logs_folder': '/opt/build/testrunner/logs/testrunner-16-Apr-06_08-41-08/test_15', 'spec': 'simple_2i_n1ql'}
[2016-04-06 08:49:53,998] - [rest_client:1636] INFO - Node versions in cluster [u'4.5.35007-1459908767-rel-enterprise']
[2016-04-06 08:49:53,998] - [basetestcase:146] INFO - ==============  basetestcase setup was started for test #15 test_delete_create_bucket_and_query==============
[2016-04-06 08:49:54,014] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:49:54,019] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:49:54,030] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:49:54,035] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:49:54,045] - [basetestcase:365] INFO - sleep for 10 secs.  ...
[2016-04-06 08:50:04,058] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9000
[2016-04-06 08:50:04,061] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9000 is running
[2016-04-06 08:50:04,064] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9001
[2016-04-06 08:50:04,067] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9001 is running
[2016-04-06 08:50:04,070] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9002
[2016-04-06 08:50:04,073] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9002 is running
[2016-04-06 08:50:04,075] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9003
[2016-04-06 08:50:04,078] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9003 is running
[2016-04-06 08:50:04,078] - [basetestcase:167] INFO - initializing cluster
[2016-04-06 08:50:05,011] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:50:05,015] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:50:05,028] - [rest_client:789] INFO - /node/controller/setupServices params on 127.0.0.1: 9000:services=kv%2Cindex%2Cn1ql&password=asdasd&hostname=127.0.0.1%3A9000&user=Administrator
[2016-04-06 08:50:05,038] - [rest_client:757] ERROR - http://127.0.0.1:9000//node/controller/setupServices error 400 reason: unknown ["cannot change node services after cluster is provisioned"]
[2016-04-06 08:50:05,038] - [rest_client:794] INFO - This node is already provisioned with services, we do not consider this as failure for test case
[2016-04-06 08:50:05,038] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9000:username=Administrator&password=asdasd&port=9000
[2016-04-06 08:50:05,057] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:50:05,061] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:50:05,067] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9001:username=Administrator&password=asdasd&port=9001
[2016-04-06 08:50:05,084] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:50:05,087] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:50:05,093] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9002:username=Administrator&password=asdasd&port=9002
[2016-04-06 08:50:05,112] - [rest_client:811] INFO - pools/default params : memoryQuota=8041
[2016-04-06 08:50:05,116] - [rest_client:830] INFO - settings/indexes params : storageMode=forestdb
[2016-04-06 08:50:05,120] - [rest_client:774] INFO - settings/web params on 127.0.0.1:9003:username=Administrator&password=asdasd&port=9003
[2016-04-06 08:50:05,133] - [basetestcase:187] INFO - done initializing cluster
[2016-04-06 08:50:06,142] - [rest_client:1772] INFO - http://127.0.0.1:9000/pools/default/buckets with param: bucketType=membase&evictionPolicy=valueOnly&threadsNumber=3&ramQuotaMB=8041&proxyPort=11211&authType=sasl&name=default&flushEnabled=1&replicaNumber=1&replicaIndex=1&saslPassword=None
[2016-04-06 08:50:06,148] - [rest_client:1794] INFO - 0.01 seconds to create bucket default
[2016-04-06 08:50:06,148] - [bucket_helper:305] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2016-04-06 08:50:07,227] - [data_helper:295] INFO - creating direct client 127.0.0.1:12000 default
[2016-04-06 08:50:07,271] - [data_helper:295] INFO - creating direct client 127.0.0.1:12000 default
[2016-04-06 08:50:07,299] - [task:288] INFO - bucket 'default' was created with per node RAM quota: 8041
[2016-04-06 08:50:07,308] - [basetestcase:257] INFO - ==============  basetestcase setup was finished for test #15 test_delete_create_bucket_and_query ==============
[2016-04-06 08:50:07,312] - [basetestcase:365] INFO - sleep for 10 secs.  ...
[2016-04-06 08:50:17,321] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:50:17,321] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:50:17,327] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:50:17,327] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:50:17,333] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:50:17,333] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:50:17,339] - [newtuq:34] INFO - Initial status of 127.0.0.1 cluster is healthy
[2016-04-06 08:50:17,339] - [newtuq:39] INFO - current status of 127.0.0.1  is healthy
[2016-04-06 08:50:17,344] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:50:17,344] - [remote_util:168] INFO - connecting to 127.0.0.1 with username : Administrator password : asdasd ssh_key: 
[2016-04-06 08:50:17,344] - [remote_util:201] INFO - Connected to 127.0.0.1
[2016-04-06 08:50:18,265] - [basetestcase:2084] INFO - create 2016 to default documents...
[2016-04-06 08:50:18,288] - [data_helper:295] INFO - creating direct client 127.0.0.1:12000 default
[2016-04-06 08:50:19,416] - [basetestcase:2095] INFO - LOAD IS FINISHED
[2016-04-06 08:50:19,426] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:50:19,426] - [newtuq:92] INFO - ip:127.0.0.1 port:9000 ssh_username:Administrator
number of buckets 1
[2016-04-06 08:50:19,427] - [tuq_helper:419] INFO - CREATE PRIMARY INDEX ON default  USING GSI
[2016-04-06 08:50:19,427] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2016-04-06 08:50:19,429] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2016-04-06 08:50:19,458] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 26.978882ms
[2016-04-06 08:50:19,458] - [tuq_helper:75] INFO - RUN QUERY CREATE PRIMARY INDEX ON default  USING GSI
[2016-04-06 08:50:19,462] - [rest_client:2508] INFO - query params : statement=CREATE+PRIMARY+INDEX+ON+default++USING+GSI
[2016-04-06 08:50:19,984] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 520.422196ms
[2016-04-06 08:50:19,984] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2016-04-06 08:50:19,987] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2016-04-06 08:50:20,002] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 13.515975ms
[2016-04-06 08:50:20,003] - [base_2i:48] INFO - {'in_between': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}, 'initial': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}, 'after': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}, 'before': {'query_explain_ops': False, 'create_index': False, 'drop_index': False, 'query_ops': False}}
[2016-04-06 08:50:20,016] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:50:20,021] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:50:20,029] - [tuq_helper:75] INFO - RUN QUERY CREATE INDEX employee5145a0d6d4564213988c6dab570294f7join_yr ON default(join_yr) USING GSI  WITH {'defer_build': True}
[2016-04-06 08:50:20,031] - [rest_client:2508] INFO - query params : statement=CREATE+INDEX+employee5145a0d6d4564213988c6dab570294f7join_yr+ON+default%28join_yr%29+USING+GSI++WITH+%7B%27defer_build%27%3A+True%7D
[2016-04-06 08:50:20,070] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 36.888563ms
[2016-04-06 08:50:20,070] - [base_2i:102] INFO - BUILD INDEX on default(employee5145a0d6d4564213988c6dab570294f7join_yr) USING GSI
[2016-04-06 08:50:20,316] - [tuq_helper:75] INFO - RUN QUERY BUILD INDEX on default(employee5145a0d6d4564213988c6dab570294f7join_yr) USING GSI
[2016-04-06 08:50:20,320] - [rest_client:2508] INFO - query params : statement=BUILD+INDEX+on+default%28employee5145a0d6d4564213988c6dab570294f7join_yr%29+USING+GSI
[2016-04-06 08:50:20,331] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 8.775996ms
[2016-04-06 08:50:20,331] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee5145a0d6d4564213988c6dab570294f7join_yr'
[2016-04-06 08:50:20,337] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee5145a0d6d4564213988c6dab570294f7join_yr%27
[2016-04-06 08:50:20,357] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 17.988965ms
[2016-04-06 08:50:21,358] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee5145a0d6d4564213988c6dab570294f7join_yr'
[2016-04-06 08:50:21,362] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee5145a0d6d4564213988c6dab570294f7join_yr%27
[2016-04-06 08:50:21,379] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 15.999966ms
[2016-04-06 08:50:22,381] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee5145a0d6d4564213988c6dab570294f7join_yr'
[2016-04-06 08:50:22,384] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee5145a0d6d4564213988c6dab570294f7join_yr%27
[2016-04-06 08:50:22,400] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 14.88493ms
[2016-04-06 08:50:22,401] - [tuq_helper:75] INFO - RUN QUERY EXPLAIN SELECT * FROM default WHERE  join_yr > 2010 and join_yr < 2014 ORDER BY _id 
[2016-04-06 08:50:22,403] - [rest_client:2508] INFO - query params : statement=EXPLAIN+SELECT+%2A+FROM+default+WHERE++join_yr+%3E+2010+and+join_yr+%3C+2014+ORDER+BY+_id+
[2016-04-06 08:50:22,408] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 3.334125ms
[2016-04-06 08:50:22,408] - [base_2i:241] INFO - {u'status': u'success', u'metrics': {u'elapsedTime': u'3.334125ms', u'executionTime': u'3.283531ms', u'resultSize': 3247, u'resultCount': 1}, u'results': [{u'text': u'SELECT * FROM default WHERE  join_yr > 2010 and join_yr < 2014 ORDER BY _id', u'plan': {u'#operator': u'Sequence', u'~children': [{u'#operator': u'Sequence', u'~children': [{u'index': u'employee5145a0d6d4564213988c6dab570294f7join_yr', u'namespace': u'default', u'keyspace': u'default', u'#operator': u'IndexScan', u'spans': [{u'Range': {u'High': [u'2014'], u'Low': [u'2010'], u'Inclusion': 0}}], u'using': u'gsi', u'index_id': u'6cc5f7da0ec601d1'}, {u'#operator': u'Parallel', u'~child': {u'#operator': u'Sequence', u'~children': [{u'keyspace': u'default', u'#operator': u'Fetch', u'namespace': u'default'}, {u'#operator': u'Filter', u'condition': u'((2010 < (`default`.`join_yr`)) and ((`default`.`join_yr`) < 2014))'}, {u'#operator': u'InitialProject', u'result_terms': [{u'expr': u'self', u'star': True}]}]}}]}, {u'#operator': u'Order', u'sort_terms': [{u'expr': u'(`default`.`_id`)'}]}, {u'#operator': u'FinalProject'}]}}], u'requestID': u'5b596af5-c2e2-4cb8-85cd-4a121e151fd5', u'signature': u'json'}
[2016-04-06 08:50:22,409] - [base_2i:284] INFO - Query : SELECT * FROM default WHERE  join_yr > 2010 and join_yr < 2014 ORDER BY _id 
[2016-04-06 08:50:22,409] - [tuq_generators:66] INFO - FROM clause ===== is default
[2016-04-06 08:50:22,409] - [tuq_generators:68] INFO - WHERE clause ===== is   doc["join_yr"] > 2010 and doc["join_yr"] < 2014 
[2016-04-06 08:50:22,409] - [tuq_generators:70] INFO - UNNEST clause ===== is None
[2016-04-06 08:50:22,409] - [tuq_generators:72] INFO - SELECT clause ===== is {"*" : doc,}
[2016-04-06 08:50:22,409] - [tuq_generators:401] INFO - ORDER clause ========= is doc["_id"],
[2016-04-06 08:50:22,410] - [tuq_generators:401] INFO - ORDER clause ========= is doc["_id"],
[2016-04-06 08:50:22,460] - [tuq_generators:401] INFO - ORDER clause ========= is doc["_id"],
[2016-04-06 08:50:22,471] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM default WHERE  join_yr > 2010 and join_yr < 2014 ORDER BY _id 
[2016-04-06 08:50:22,474] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+default+WHERE++join_yr+%3E+2010+and+join_yr+%3C+2014+ORDER+BY+_id+
[2016-04-06 08:50:22,806] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 310.758649ms
[2016-04-06 08:50:22,855] - [tuq_helper:108] INFO -  Analyzing Actual Result
[2016-04-06 08:50:22,856] - [tuq_helper:110] INFO -  Analyzing Expected Result
[2016-04-06 08:50:24,408] - [basetestcase:365] INFO - sleep for 2 secs.  ...
[2016-04-06 08:50:27,347] - [rest_client:1772] INFO - http://127.0.0.1:9000/pools/default/buckets with param: bucketType=membase&evictionPolicy=valueOnly&threadsNumber=3&ramQuotaMB=8041&proxyPort=11211&authType=sasl&name=default&flushEnabled=1&replicaNumber=1&replicaIndex=1&saslPassword=None
[2016-04-06 08:50:27,351] - [rest_client:1794] INFO - 0.00 seconds to create bucket default
[2016-04-06 08:50:27,352] - [bucket_helper:305] INFO - waiting for memcached bucket : default in 127.0.0.1 to accept set ops
[2016-04-06 08:50:28,409] - [data_helper:295] INFO - creating direct client 127.0.0.1:12000 default
[2016-04-06 08:50:28,445] - [data_helper:295] INFO - creating direct client 127.0.0.1:12000 default
[2016-04-06 08:50:28,470] - [task:288] INFO - bucket 'default' was created with per node RAM quota: 8041
[2016-04-06 08:50:28,479] - [basetestcase:365] INFO - sleep for 2 secs.  ...
[2016-04-06 08:50:30,485] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9000
[2016-04-06 08:50:30,488] - [rest_client:43] WARNING - server 127.0.0.1:9000 status is warmup
[2016-04-06 08:50:31,492] - [rest_client:43] WARNING - server 127.0.0.1:9000 status is warmup
[2016-04-06 08:50:32,496] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9000 is running
[2016-04-06 08:50:32,499] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9001
[2016-04-06 08:50:32,502] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9001 is running
[2016-04-06 08:50:32,504] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9002
[2016-04-06 08:50:32,507] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9002 is running
[2016-04-06 08:50:32,510] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9003
[2016-04-06 08:50:32,512] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9003 is running
[2016-04-06 08:50:32,517] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:50:32,518] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee5145a0d6d4564213988c6dab570294f7join_yr'
[2016-04-06 08:50:32,524] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee5145a0d6d4564213988c6dab570294f7join_yr%27
[2016-04-06 08:50:32,540] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 14.658979ms
[2016-04-06 08:50:32,541] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = 'employee5145a0d6d4564213988c6dab570294f7join_yr'
[2016-04-06 08:50:32,544] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27employee5145a0d6d4564213988c6dab570294f7join_yr%27
[2016-04-06 08:50:32,565] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 19.163063ms
[2016-04-06 08:50:32,570] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:50:32,583] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:50:32,583] - [remote_util:168] INFO - connecting to 127.0.0.1 with username : Administrator password : asdasd ssh_key: 
[2016-04-06 08:50:32,583] - [remote_util:201] INFO - Connected to 127.0.0.1
[2016-04-06 08:50:32,587] - [rest_client:1222] INFO - /diag/eval status on 127.0.0.1:9000: True content: "/opt/build/ns_server/logs/n_0" command: filename:absname(element(2, application:get_env(ns_server,error_logger_mf_dir))).
[2016-04-06 08:50:32,764] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: zgrep "panic" "/opt/build/ns_server/logs/n_0"/indexer.log* | wc -l
[2016-04-06 08:50:32,794] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:50:32,801] - [basetestcase:1981] INFO - list of all nodes in cluster: [ip:127.0.0.1 port:9000 ssh_username:Administrator]
[2016-04-06 08:50:32,801] - [tuq_helper:392] INFO - CHECK FOR PRIMARY INDEXES
[2016-04-06 08:50:32,801] - [tuq_helper:399] INFO - DROP PRIMARY INDEX ON default USING GSI
[2016-04-06 08:50:32,801] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2016-04-06 08:50:32,804] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2016-04-06 08:50:32,824] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 17.889278ms
[2016-04-06 08:50:32,824] - [tuq_helper:399] INFO - DROP PRIMARY INDEX ON default USING GSI
[2016-04-06 08:50:32,824] - [tuq_helper:75] INFO - RUN QUERY SELECT * FROM system:indexes where name = '#primary'
[2016-04-06 08:50:32,827] - [rest_client:2508] INFO - query params : statement=SELECT+%2A+FROM+system%3Aindexes+where+name+%3D+%27%23primary%27
[2016-04-06 08:50:32,839] - [tuq_helper:104] INFO - TOTAL ELAPSED TIME: 11.287694ms
[2016-04-06 08:50:33,011] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: rm -f /opt/couchbase/var/lib/couchbase/meta;rm -f /tmp/log_upr_client.sock
[2016-04-06 08:50:33,037] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:50:33,038] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: killall cbq-engine
[2016-04-06 08:50:33,068] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:50:33,068] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: killall tuqtng
[2016-04-06 08:50:33,103] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:50:33,104] - [remote_util:2279] INFO - running command.raw on 127.0.0.1: killall indexer
[2016-04-06 08:50:33,146] - [remote_util:2316] INFO - command executed successfully
[2016-04-06 08:50:33,147] - [basetestcase:297] INFO - ==============  basetestcase cleanup was started for test #15 test_delete_create_bucket_and_query ==============
[2016-04-06 08:50:33,173] - [bucket_helper:137] INFO - deleting existing buckets [u'default'] on 127.0.0.1
[2016-04-06 08:50:33,173] - [bucket_helper:139] INFO - remove bucket default ...
[2016-04-06 08:50:33,613] - [bucket_helper:153] INFO - deleted bucket : default from 127.0.0.1
[2016-04-06 08:50:33,614] - [bucket_helper:226] INFO - waiting for bucket deletion to complete....
[2016-04-06 08:50:33,619] - [rest_client:127] INFO - node 127.0.0.1 existing buckets : []
[2016-04-06 08:50:33,624] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:50:33,630] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:50:33,635] - [bucket_helper:137] INFO - deleting existing buckets [] on 127.0.0.1
[2016-04-06 08:50:33,648] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9000
[2016-04-06 08:50:33,652] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9000 is running
[2016-04-06 08:50:33,655] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9001
[2016-04-06 08:50:33,658] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9001 is running
[2016-04-06 08:50:33,660] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9002
[2016-04-06 08:50:33,663] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9002 is running
[2016-04-06 08:50:33,666] - [cluster_helper:78] INFO - waiting for ns_server @ 127.0.0.1:9003
[2016-04-06 08:50:33,668] - [cluster_helper:80] INFO - ns_server @ 127.0.0.1:9003 is running
[2016-04-06 08:50:33,669] - [basetestcase:315] INFO - ==============  basetestcase cleanup was finished for test #15 test_delete_create_bucket_and_query ==============
ok

----------------------------------------------------------------------
Ran 1 test in 39.682s

OK
Cluster instance shutdown with force
summary so far suite 2i.indexscans_2i.SecondaryIndexingScanTests , pass 9 , fail 0
summary so far suite 2i.indexcreatedrop_2i.SecondaryIndexingCreateDropTests , pass 2 , fail 0
summary so far suite 2i.cluster_ops_2i.SecondaryIndexingClusterOpsTests , pass 4 , fail 0
testrunner logs, diags and results are available under /opt/build/testrunner/logs/testrunner-16-Apr-06_08-41-08/test_15
Run after suite setup for 2i.cluster_ops_2i.SecondaryIndexingClusterOpsTests.test_delete_create_bucket_and_query
2i.indexscans_2i.SecondaryIndexingScanTests.test_multi_create_query_explain_drop_index  pass
2i.indexscans_2i.SecondaryIndexingScanTests.test_multi_create_query_explain_drop_index  pass
2i.indexscans_2i.SecondaryIndexingScanTests.test_multi_create_query_explain_drop_index  pass
2i.indexscans_2i.SecondaryIndexingScanTests.test_multi_create_query_explain_drop_index  pass
2i.indexscans_2i.SecondaryIndexingScanTests.test_multi_create_query_explain_drop_index  pass
2i.indexscans_2i.SecondaryIndexingScanTests.test_multi_create_query_explain_drop_index  pass
2i.indexscans_2i.SecondaryIndexingScanTests.test_multi_create_query_explain_drop_index  pass
2i.indexscans_2i.SecondaryIndexingScanTests.test_multi_create_query_explain_drop_index  pass
2i.indexscans_2i.SecondaryIndexingScanTests.test_multi_create_query_explain_drop_index  pass
2i.indexcreatedrop_2i.SecondaryIndexingCreateDropTests.test_multi_create_drop_index  pass
2i.indexcreatedrop_2i.SecondaryIndexingCreateDropTests.test_multi_create_drop_index  pass
2i.cluster_ops_2i.SecondaryIndexingClusterOpsTests.test_remove_bucket_and_query  pass
2i.cluster_ops_2i.SecondaryIndexingClusterOpsTests.test_change_bucket_properties  pass
2i.cluster_ops_2i.SecondaryIndexingClusterOpsTests.test_flush_bucket_and_query  pass
2i.cluster_ops_2i.SecondaryIndexingClusterOpsTests.test_delete_create_bucket_and_query  pass
Thread Cluster_Thread was not properly terminated, will be terminated now.
scripts/start_cluster_and_run_tests.sh: line 81: 13288 Terminated              COUCHBASE_NUM_VBUCKETS=64 python ./cluster_run --nodes=$servers_count &> $wd/cluster_run.log  (wd: /opt/build/ns_server)

Testing Failed: Required test failed

FAIL	github.com/couchbase/indexing/secondary/tests/functionaltests	0.218s

Version: versions-06.04.2016-07.42.cfg
Build Log: make-06.04.2016-07.42.log
Server Log: logs-06.04.2016-07.42.tar.gz
Collect Info: collects-06.04.2016-07.42.tar
Install: installer-06.04.2016-07.42.tar

Finished